Add a key update notification #1144

jjz921024 · 2024-10-10T12:56:17Z

this pr want to close #1143

ranshid · 2024-10-10T14:53:34Z

src/server.c

@@ -3537,6 +3537,10 @@ void call(client *c, int flags) {
    dirty = server.dirty - dirty;
    if (dirty < 0) dirty = 0;

+    if (dirty > 0) {
+        notifyKeyspaceEvent(NOTIFY_DIRTY, "dirty", c->argv[1], c->db->id);


This will not target specific key, but only indicate the dataset is dirty, there are many multi key commands (like mset, eval etc...) which will make many keys dirty. I think maybe a better place is to place it in signalModifiedKey

Thank you for your guidance. I had put it in signalModifiedKey.

ranshid · 2024-10-10T14:56:02Z

src/notify.c

@@ -58,6 +58,7 @@ int keyspaceEventsStringToFlags(char *classes) {
        case 'm': flags |= NOTIFY_KEY_MISS; break;
        case 'd': flags |= NOTIFY_MODULE; break;
        case 'n': flags |= NOTIFY_NEW; break;
+        case 'c': flags |= NOTIFY_DIRTY; break;


Personally I do not find the 'dirty' as the correct state here. IMO dirty means that the key was modified and was not yet consisted (like for example successfully written to the replica).
Although funny name but maybe: NOTIFY_MODIFY is a better match for what really happened?
Also maybe NOTIFY_UPDATE?

I perfer NOTIFY_UPDATE, because we can use 'u' to stand it. (The letter 'm' has been used)

ranshid · 2024-10-10T15:04:15Z

src/config.c

@@ -2906,7 +2906,7 @@ static int setConfigNotifyKeyspaceEventsOption(standardConfig *config, sds *argv
    }
    int flags = keyspaceEventsStringToFlags(argv[0]);
    if (flags == -1) {
-        *err = "Invalid event class character. Use 'Ag$lshzxeKEtmdn'.";
+        *err = "Invalid event class character. Use 'Ag$lshzxeKEtmdnc'.";


why 'c'? maybe 'u' (stands for updated)?

ranshid · 2024-10-10T15:19:29Z

@jjz921024 you will have to sign your commit in order for the DCO to pass.

Signed-off-by: anotherJJz <[email protected]>

ranshid · 2024-10-13T06:31:44Z

@valkey-io/core-team since this is an interface/config change, tagged it as major decision. can you please take a look?

codecov · 2024-10-13T14:24:13Z

Codecov Report

Attention: Patch coverage is 75.00000% with 1 line in your changes missing coverage. Please review.

Project coverage is 70.63%. Comparing base (9b8a061) to head (dcdcfb1).
Report is 5 commits behind head on unstable.

Files with missing lines	Patch %	Lines
src/notify.c	50.00%	1 Missing ⚠️

Additional details and impacted files

@@             Coverage Diff              @@
##           unstable    #1144      +/-   ##
============================================
- Coverage     70.67%   70.63%   -0.04%     
============================================
  Files           114      114              
  Lines         61717    61678      -39     
============================================
- Hits          43617    43568      -49     
- Misses        18100    18110      +10

Files with missing lines	Coverage Δ
src/config.c	`78.64% <100.00%> (-0.06%)`	⬇️
src/db.c	`88.42% <100.00%> (ø)`
src/server.h	`100.00% <ø> (ø)`
src/notify.c	`95.65% <50.00%> (-1.37%)`	⬇️

... and 43 files with indirect coverage changes

PingXie · 2024-10-14T04:24:01Z

@valkey-io/core-team since this is an interface/config change, tagged it as major decision. can you please take a look?

How is this new setting different from g$lshzxet (or just g$lshzxe if stream events are not interesting)? Are there any specific updates that are not covered by the existing event types? The way it is implemented now seems to be a super set of the existing event types.

ranshid · 2024-10-15T14:06:42Z

@valkey-io/core-team since this is an interface/config change, tagged it as major decision. can you please take a look?

How is this new setting different from g$lshzxet (or just g$lshzxe if stream events are not interesting)? Are there any specific updates that are not covered by the existing event types? The way it is implemented now seems to be a super set of the existing event types.

I agree. this will basically aggregate different events under the same event name. mainly to simplify the client side logic and can be achieved without this. However that got me thinking if we see any value to provide some "update" notification indicating an existing value was changed (as apposed to created/deleted/expired etc...). however that would require a different change introduced.

jjz921024 · 2024-10-15T15:08:11Z

@valkey-io/core-team since this is an interface/config change, tagged it as major decision. can you please take a look?

How is this new setting different from g$lshzxet (or just g$lshzxe if stream events are not interesting)? Are there any specific updates that are not covered by the existing event types? The way it is implemented now seems to be a super set of the existing event types.

@PingXie Hi, It might be different with g$lshzxet. The existing event type does not contain events for which the key is modified. Please let me know if I'm wrong.

jjz921024 · 2024-10-15T15:17:52Z

@valkey-io/core-team since this is an interface/config change, tagged it as major decision. can you please take a look?

How is this new setting different from g$lshzxet (or just g$lshzxe if stream events are not interesting)? Are there any specific updates that are not covered by the existing event types? The way it is implemented now seems to be a super set of the existing event types.

I agree. this will basically aggregate different events under the same event name. mainly to simplify the client side logic and can be achieved without this. However that got me thinking if we see any value to provide some "update" notification indicating an existing value was changed (as apposed to created/deleted/expired etc...). however that would require a different change introduced.

@ranshid Hi, It will aggregate different events under the same event name. But, if we want to provide update notification when a key has been modify. Maybe we should introduce it.

PingXie · 2024-10-15T15:38:33Z

The existing event type does not contain events for which the key is modified. Please let me know if I'm wrong

I assume you meant "value modification" rather than "key modification" here? The current key notification mechanism breaks events down by data/command types like SET, HSET, etc. From my quick look at the source code, it appears to cover all relevant operations. Are you seeing any "update" operations being missed? I'm cautious about introducing another "superset" event, as it might add unnecessary complexity/overhead.

BTW, we do have "rename" (or "key modification") events too.

ranshid · 2024-10-15T17:09:43Z

BTW, we do have "rename" (or "key modification") events too.

Yes we do. we have rename_from and rename_to events

ranshid · 2024-10-15T17:13:33Z

The existing event type does not contain events for which the key is modified. Please let me know if I'm wrong

I assume you meant "value modification" rather than "key modification" here? The current key notification mechanism breaks events down by data/command types like SET, HSET, etc. From my quick look at the source code, it appears to cover all relevant operations. Are you seeing any "update" operations being missed? I'm cautious about introducing another "superset" event, as it might add unnecessary complexity/overhead.

I think, as @PingXie correctly stated, that the current events ARE reflecting most of the mutative data commands. However currently there is not easy way for the client to understand a VALUE was changed other then combining the "new" notification and all other different events which are per operation/datatype. I think it might simplify the client logic to be able to have event notification for "modified" value but as I said this IS currently possible to achieve by clients IMO.

ranshid reviewed Oct 10, 2024

View reviewed changes

Add a key update notification

718b0ff

Signed-off-by: anotherJJz <[email protected]>

jjz921024 force-pushed the dirty branch from 12f1377 to 718b0ff Compare October 11, 2024 03:07

jjz921024 changed the title ~~Add a dirty key notification~~ Add a key update notification Oct 11, 2024

fix comment

dcdcfb1

Signed-off-by: anotherJJz <[email protected]>

jjz921024 requested a review from ranshid October 12, 2024 01:57

ranshid added the major-decision-pending Major decision pending by TSC team label Oct 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add a key update notification #1144

Add a key update notification #1144

jjz921024 commented Oct 10, 2024

ranshid Oct 10, 2024 •

edited

Loading

jjz921024 Oct 11, 2024

ranshid Oct 10, 2024 •

edited

Loading

jjz921024 Oct 11, 2024 •

edited

Loading

ranshid Oct 10, 2024

ranshid commented Oct 10, 2024

ranshid commented Oct 13, 2024 •

edited

Loading

codecov bot commented Oct 13, 2024 •

edited

Loading

PingXie commented Oct 14, 2024

ranshid commented Oct 15, 2024

jjz921024 commented Oct 15, 2024 •

edited

Loading

jjz921024 commented Oct 15, 2024

PingXie commented Oct 15, 2024

ranshid commented Oct 15, 2024

ranshid commented Oct 15, 2024

Add a key update notification #1144

Are you sure you want to change the base?

Add a key update notification #1144

Conversation

jjz921024 commented Oct 10, 2024

ranshid Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

jjz921024 Oct 11, 2024

Choose a reason for hiding this comment

ranshid Oct 10, 2024 • edited Loading

Choose a reason for hiding this comment

jjz921024 Oct 11, 2024 • edited Loading

Choose a reason for hiding this comment

ranshid Oct 10, 2024

Choose a reason for hiding this comment

ranshid commented Oct 10, 2024

ranshid commented Oct 13, 2024 • edited Loading

codecov bot commented Oct 13, 2024 • edited Loading

Codecov Report

PingXie commented Oct 14, 2024

ranshid commented Oct 15, 2024

jjz921024 commented Oct 15, 2024 • edited Loading

jjz921024 commented Oct 15, 2024

PingXie commented Oct 15, 2024

ranshid commented Oct 15, 2024

ranshid commented Oct 15, 2024

ranshid Oct 10, 2024 •

edited

Loading

ranshid Oct 10, 2024 •

edited

Loading

jjz921024 Oct 11, 2024 •

edited

Loading

ranshid commented Oct 13, 2024 •

edited

Loading

codecov bot commented Oct 13, 2024 •

edited

Loading

jjz921024 commented Oct 15, 2024 •

edited

Loading