-
Notifications
You must be signed in to change notification settings - Fork 9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Issue 3489 - Upgrade to Java 17 #3499
base: 2000-upgrade-emr
Are you sure you want to change the base?
Conversation
2d84631
to
5272c1c
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Passed awaiting testing completion
Bulk import on EKS is failing on this branch currently. It passes on EMR, on both serverless and EC2. |
It looks like the problem is the Spark Docker image has the wrong version of Hadoop in it. The Spark Docker image includes Hadoop 3.3.4, but EMR and Sleeper use Hadoop 3.3.6. The Spark Docker image doesn't include the hadoop-aws module. We're bringing that in in Sleeper and the versions conflict. It looks as though the class IOStatisticsBinding has been moved between modules and it can't find it. If we want to fix it we'll probably need a new Docker image. |
924efd2
to
b57df79
Compare
Make sure you have checked all steps below.
Issue
PR"
Tests
Documentation
separate issue for that below.