Table of Contents

Shortcuts

thunder.distributed.FSDPType¶

class thunder.distributed.FSDPType(value, names=None, *, module=None, qualname=None, type=None, start=1, boundary=None)[source]¶

Bases: Enum

Specifies the sharding strategy to be used for FSDP in Thunder.

ZERO2[source]¶: Similar to torch.distributed.fsdp.ShardingStrategy.SHARD_GRAD_OP.

ZERO3[source]¶: Similar to torch.distributed.fsdp.ShardingStrategy.FULL_SHARD.

__init__(*args, **kwds)[source]¶

Attributes

`ZERO2`
`ZERO3`