[PROBLEM]

[jeisner]

Identify the file to be fixed
examples/Chat_finetuning_data_prep.ipynb

Describe the problem
Raises exception if the fine-tuning messages include "weight" keys.

Describe a solution
The problem is here:

for key, value in message.items():
    num_tokens += len(encoding.encode(value))

If key=='weight', you should not call encode(value), because value should be numeric.
(You may want to check that value is numeric and indeed that value in [0,1], which is all that is supported now, at least according to OpenAI's documentation.)

[harshaygadekar]

Actually yes the "weight" value is expected to be a numeric value between 0 and 1, not a string to be encoded.
How about raising a "ValueError" if the weight is not a number between 0 and 1, which also aligns with OpenAI's current documentation.

for key, value in message.items():
            if key == "weight":
                if not isinstance(value, (int, float)) or not (0 <= value <= 1):
                    raise ValueError(f"Weight must be a number between 0 and 1, got: {value}")
            else:
                num_tokens += len(encoding.encode(str(value)))

[jeisner]

Yes, like that.
Except that it should be value in [0,1] (or value==0 or value==1) rather than 0 <= value <= 1.
OpenAI's documentation says:

The allowed values for weight are currently 0 or 1.

Or do you think the documentation is wrong and 0.5 is allowed? I haven't tried it.

It would be nice if they allowed it to be any real number, and maybe they will in future.

[harshaygadekar]

I haven't tried 0.5 yet either, but I plan to give it a shot.
Maybe It's possible they haven't updated the documentation. And yes it would certainly be more convenient if they allowed any real number to be used.

[github-actions]

This issue is stale because it has been open 60 days with no activity. Remove stale label or comment or this will be closed in 10 days.

[jeisner]

I don't think I have the power to remove the Stale label, so commenting to keep this issue open.