accelerate
Parallelism config + TP + HSDP + BYODM (Bring Your Own Device Mesh)
#3682
Merged
Overview
Commits
76
Changes
View On GitHub
Commits
  • Feat: init
    S1ro1 committed 1 year ago
  • Feat: add validation + init from kwargs
    S1ro1 committed 1 year ago
  • Fix: minor fixes
    S1ro1 committed 1 year ago
  • Feat: more cleanup
    S1ro1 committed 1 year ago
  • Minor refactor
    S1ro1 committed 1 year ago
  • remove import
    S1ro1 committed 1 year ago
  • adding support for pre-configured device mesh
    salmanmohammadi committed 1 year ago
  • adding device mesh to fsdp2
    salmanmohammadi committed 1 year ago
  • moving mesh dim defn to parralismconfig
    salmanmohammadi committed 1 year ago
  • tests
    salmanmohammadi committed 1 year ago
  • WIP device mesh/accelerator validation
    salmanmohammadi committed 1 year ago
  • WIP more tests
    salmanmohammadi committed 1 year ago
  • Test Driven Development (TDD)
    salmanmohammadi committed 1 year ago
  • fixing build_device_mesh
    salmanmohammadi committed 1 year ago
  • FSDP dim names
    salmanmohammadi committed 1 year ago
  • adding example
    Salman Mohammadi committed 1 year ago
  • WIP
    Salman Mohammadi committed 1 year ago
  • fixing HSDP
    Salman Mohammadi committed 1 year ago
  • Feat: add back old options
    S1ro1 committed 1 year ago
  • working example
    Salman Mohammadi committed 1 year ago
  • debugging
    Salman Mohammadi committed 1 year ago
  • adding parallelism config to partialstate
    Salman Mohammadi committed 1 year ago
  • Feat: revert ddp changes
    S1ro1 committed 1 year ago
  • Revert DDP
    S1ro1 committed 1 year ago
  • Feat: (untested) update mesh dims and some minor tweaks
    S1ro1 committed 1 year ago
  • adding dp_cp dims
    Salman Mohammadi committed 1 year ago
  • updating comments
    Salman Mohammadi committed 1 year ago
  • WIP
    Salman Mohammadi committed 1 year ago
  • wip 2
    Salman Mohammadi committed 1 year ago
  • reverting
    Salman Mohammadi committed 1 year ago
  • storing state in accelerator rather than acceleratorstate
    Salman Mohammadi committed 1 year ago
  • Fix: minor tweaks
    S1ro1 committed 1 year ago
  • wip example update
    Salman Mohammadi committed 1 year ago
  • merging
    Salman Mohammadi committed 1 year ago
  • Fixes for non-fsdp2 case
    S1ro1 committed 1 year ago
  • Feat: ensure ddp/tp only works
    S1ro1 committed 1 year ago
  • updating example
    Salman Mohammadi committed 1 year ago
  • updating example
    Salman Mohammadi committed 1 year ago
  • updating examples, fixing state
    Salman Mohammadi committed 1 year ago
  • fixed state
    Salman Mohammadi committed 1 year ago
  • comments
    Salman Mohammadi committed 1 year ago
  • fixing partial state check
    Salman Mohammadi committed 1 year ago
  • linting
    Salman Mohammadi committed 1 year ago
  • comments
    Salman Mohammadi committed 1 year ago
  • removing fn
    Salman Mohammadi committed 1 year ago
  • merging
    Salman Mohammadi committed 1 year ago
  • WIP: fix tp
    S1ro1 committed 1 year ago
  • comments
    Salman Mohammadi committed 1 year ago
  • removing return
    Salman Mohammadi committed 1 year ago
  • reverting upcast
    Salman Mohammadi committed 1 year ago
  • add guards
    winglian committed 1 year ago
  • guards for empty self.parallelism_config
    winglian committed 1 year ago
  • use len on tuple to check if empty
    winglian committed 1 year ago
  • Feat: cleanup example
    S1ro1 committed 1 year ago
  • Feat: some cleanup of example
    S1ro1 committed 1 year ago
  • Merge branch 'main' into device_mesh_parallelism_config
    S1ro1 committed 1 year ago
  • Feat: add trackio
    S1ro1 committed 1 year ago
  • Fix: improve trackio
    S1ro1 committed 1 year ago
  • Feat: TP works
    S1ro1 committed 1 year ago
  • Feat: some fsdp2 improv
    S1ro1 committed 1 year ago
  • Feat: working examples
    S1ro1 committed 1 year ago
  • handle clipping for tensor parallel
    winglian committed 1 year ago
  • Implicit replicate
    S1ro1 committed 1 year ago
  • Refactor: move to separate file + cleanup + basic comments
    S1ro1 committed 1 year ago
  • Fix: add unadded files, fix circular import
    S1ro1 committed 1 year ago
  • Feat: better readme
    S1ro1 committed 1 year ago
  • Feat: add blog + ultrascale links
    S1ro1 committed 1 year ago
  • Tmp: should_save_model now returns only true
    S1ro1 committed 1 year ago
  • Fix: remove implicit_replication and style
    S1ro1 committed 1 year ago
  • Fix: remove optional
    S1ro1 committed 1 year ago
  • add guard on parallelism_config.tp_enabled
    winglian committed 1 year ago
  • fix import
    winglian committed 1 year ago
  • fixing empty parallelism_config
    Salman Mohammadi committed 1 year ago
  • fix import path for test patch
    winglian committed 1 year ago
  • fixing patch
    Salman Mohammadi committed 1 year ago
  • merging
    Salman Mohammadi committed 1 year ago
FAQ Terms Privacy Refunds Impressum
Loading