[venice-server] Dropping unassigned partitions

[kristyelee]

Summary, imperative, start upper case, don't end with a period

The goal is to remove unassigned partitions that are on disk state

This involves storage service and engine, which upon starting (constructed), will go through folders on disk and verify status of folders
Helix's ideal state is used for consulting partitions and their assignment
The change proposed includes defining a function in VeniceServer that goes through partitions in idealState and compares to partitions in storageEngine
StorageEngine object initialization is also being updated to run the new function that performs this verification.

Resolves #650

How was this PR tested?

Will write corresponding test.
EDIT: A unit test has been written.

Does this PR introduce any user-facing changes?

• No. You can skip the rest of this section.
• Yes. Make sure to explain your proposed changes and call out the behavior change.

[kvargha]

It looks like spotBugs is failing.

If you run this locally, you'll be able to read the report to see what's wrong.
./gradlew :services:venice-server:spotbugsMain

Make sure to build before running this.

[kristyelee]

(Suggested addition to code, that seems directly useful but is no longer / not used actively:

In the HelixParticipationService file, similar to the getInstance function, I also wrote a getHelixManager function. I tried using this before in access after HelixParticipationService is instantiated, but have changed this implementation. )

(Reverted code in this file back to original state but thought I should make a note of this).

[ZacAttack]

This also isn't quite right. The ideal state is the superset of all partitions for the store. All partitions ID's in the storageEngine will be in the superset.

We need to parse more information from the ideal state. Specifically, for each partition that is on this node, we need to determine if the ideal state for that partition id contains the node name.

To better paint the picture, an ideal state is a big json document that looks like this:

"Test_Store_Migration_Demo_v1_0": {
"lor1-app56585.prod.linkedin.com_1690": "LEADER",
"lor1-app56614.prod.linkedin.com_1690": "STANDBY",
"lor1-app110448.prod.linkedin.com_1690": "STANDBY"
},
"Test_Store_Migration_Demo_v1_1": {
"lor1-app56586.prod.linkedin.com_1690": "LEADER",
"lor1-app71895.prod.linkedin.com_1690": "STANDBY",
"lor1-app111181.prod.linkedin.com_1690": "STANDBY"
},

Here, for partitions 1 and 0, we have three hosts assigned roles for that partition. What we should do is, does the current host we're working with reside in these lists?

[kristyelee]

I have written the comparison and retrieved the current host from manager.getInstanceName().

[kristyelee]

I wrote the related unit test for this PR.
GitHub tests pass for program, except for one Integration test unrelated to my change.