HIVE-28538: Post JDK11-Fix locale-related discrepancies in test cases between local and Jenkins environments #5476
Conversation
… between local and Jenkins environments
|
| @@ -92,7 +92,7 @@ | |||
| <test.excludes.additional/> | |||
| <!-- Plugin and Plugin Dependency Versions --> | |||
| <ant.contrib.version>1.0b3</ant.contrib.version> | |||
| <maven.test.jvm.args>-Xmx2048m -DJETTY_AVAILABLE_PROCESSORS=4</maven.test.jvm.args> | |||
| <maven.test.jvm.args>-Xmx2048m -DJETTY_AVAILABLE_PROCESSORS=4 -Duser.country=US</maven.test.jvm.args> | |||
There was a problem hiding this comment.
I see that in the configuration of the maven-surefile-plugin we have the following:
<environmentVariables>
<TZ>US/Pacific</TZ>
<LANG>en_US.UTF-8</LANG>
...
</environmentVariables>
Shouldn't this be enough to guarantee that there are no discrepancies. Why do we need to set also the JVM properties?
There was a problem hiding this comment.
The existing configuration in the maven-surefire-plugin indeed sets the LANG environment variable to en_US.UTF-8, which helps ensure that environment variables are set correctly within the Jenkins build environment. However, this setting does not fully address the locale discrepancies that can occur when tests are run locally on different developer machines with varying locale settings.
Environment variables can influence the locale settings for the system environment but might not comprehensively cover all locale-specific behaviors within the JVM. Passing the locale explicitly via <maven.test.jvm.args> ensures that the JVM itself runs with the specified locale, aligning both local and Jenkins environments more reliably.
There was a problem hiding this comment.
Isn't the JVM picking up the default values from the respective environment variables?
The Java Virtual Machine sets the default locale during startup based on the host environment.
Are you saying that locally the environmentVariables do not work as expected?
There was a problem hiding this comment.
JVM is picking up the timezone but not the locale as in the environment we are setting the TZ
Locale influences how data is formatted and presented, such as dates and times, but does not determine the actual time.
Timezone determines the current time based on the region's offset from UTC but does not affect how dates and times are formatted
Thus the test are failing with the formatting problem of am/AM or pm/PM
|
@zabetak Thanks for the review, I have addressed the comment |
What changes were proposed in this pull request?
Passing the locale in the pom.xml as <maven.test.jvm.args> to align both environments of local and jenkins
Why are the changes needed?
Post jdk11: In JDK11, changes were made to the Java locale handling as documented in the following issues:
(https://bugs.openjdk.org/browse/JDK-8145136)
https://bugs.openjdk.org/browse/JDK-8211985
These changes result in different locale behaviors. This issue does not occur in Jenkins because the test cases are written according to the en_US locale. However, when run locally, the locale may differ (e.g., en_IN), causing discrepancies. For instance, en_IN would expect am/pm while the tests expect AM/PM, leading to failures.
The changes are needed to address test failures that occur locally but not in Jenkins. This discrepancy arises due to different locale settings between the two environments.
Various tests will start failing in local without alarming in the jenkins for example TestMiniLlapLocalCliDriver.udf_date_format.q as the locale in jenkins is en_US whereas varying from the user locale. ( for example en_IN ), the test will start failing
There may be other test failures not yet captured due to different locales in local environments. Ensure all tests run with the en_US locale to identify and resolve any further issues.
Does this PR introduce any user-facing change?
NO