Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow passing uninitialized Accelerator when using AccelerateMixin #960

Open
BenjaminBossan opened this issue Apr 28, 2023 · 0 comments
Open

Comments

@BenjaminBossan
Copy link
Collaborator

Right now, when using AccelerateMixin, the instantiated Accelerator instance needs to be passed:

accelerator = Accelerator(...)
net = AcceleratedNet(
    MyModule,
    accelerator=accelerator,
)
net.fit(X, y)

This new feature should allow to pass the Accelerator class directly, with possible extra arguments using the __ notation:

net = AcceleratedNet(
    MyModule,
    accelerator=Accelerator,
    accelerator__mixed_precision='fp16',
)
net.fit(X, y)

The change should be implemented in a backwards compatible manner, so that passing the instantiated Accelerator keeps on working.

The reason for this change is not so much that we want to, for instance, grid search on the Accelerator parameters, which makes little sense. Mainly, it would be for consistency, so that the Accelerator can be instantiated and configured in the same way as other components like the module and optimizer.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant