You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the alignment-handbook, we implemented a "dataset mixer" that allows one to easily combine datasets in varying proportions, provided they all share the same schema.
It could be interesting to port this mixer to TRL, so that users can easily combine datasets during training. The only caveat I see is that to support the CLI training e.g. trl sft ... we'd need a data structure that is compatible because dict objects don't place nice with CLIs.
Motivation
Advanced post-training typically combines different datasets / proportions. Supporting this in TRL would allow us to gradually deprecate the handbook in favour of using the lib directly.
Your contribution
Open to discussion :)
The text was updated successfully, but these errors were encountered:
Feature request
In the
alignment-handbook
, we implemented a "dataset mixer" that allows one to easily combine datasets in varying proportions, provided they all share the same schema.It could be interesting to port this mixer to TRL, so that users can easily combine datasets during training. The only caveat I see is that to support the CLI training e.g.
trl sft ...
we'd need a data structure that is compatible becausedict
objects don't place nice with CLIs.Motivation
Advanced post-training typically combines different datasets / proportions. Supporting this in TRL would allow us to gradually deprecate the handbook in favour of using the lib directly.
Your contribution
Open to discussion :)
The text was updated successfully, but these errors were encountered: