text-generation-inference

MLPSpeculator.

#1865

Merged

added a bunch of cleanup based on comments in PR; removed conditionals from LayerNormParameterized and renamed to MLPSpeculatorLayerNorm; now using modules for tensor-parallel (this is work in progress - looking into if this is right approach); fixed issue with getting medusa model; fixed for more efficient loading

Hardcode a few stuff to make it work.

[REWRITTEN] added a bunch of cleanup based on comments in PR; removed conditionals from LayerNormParameterized and renamed to MLPSpeculatorLayerNorm; now using modules for tensor-parallel (this is work in progress - looking into if this is right approach); fixed issue with getting medusa model; fixed for more efficient loading

Fixed speculator.

Cleanup.

Removed a bunch of hardcodes.

Rebase after refactor.

Missing update after rebase

Remove traces of `use_medusa`.

Backport changes in medusa.

Remove old code again.

Model_type location.

Reload model_type when speculator is found.