PR #7191 Propagate Cast operations to maximize lower precision (float16) computation

satyajandhyala added component:ortmodule

satyajandhyala force pushed from 7c8346fb to 1512ec31 5 years ago

SherlockNoMad commented on 2021-04-01

satyajandhyala closed this 5 years ago

satyajandhyala reopened this 5 years ago

satyajandhyala force pushed from 1512ec31 to cbe7deb6 5 years ago

ytaous commented on 2021-04-02

satyajandhyala closed this 5 years ago

satyajandhyala reopened this 5 years ago

satyajandhyala force pushed from 87d4b277 to ca6c7624 5 years ago

satyajandhyala force pushed from 01c1e948 to d7b37a64 5 years ago

satyajandhyala force pushed from 5a38c659 to e4a244c4 5 years ago

satyajandhyala changed the title ~~[WIP] Propagate Cast operations to maximize lower precision (float16) computation~~ Propagate Cast operations to maximize lower precision (float16) computation 5 years ago

satyajandhyala marked this pull request as ready for review 5 years ago

satyajandhyala requested a review from

baijumeswani 5 years ago

satyajandhyala requested a review from

BowenBao 5 years ago

satyajandhyala requested a review from

liqunfu 5 years ago

satyajandhyala requested a review from

spandantiwari 5 years ago

satyajandhyala requested a review from

thiagocrepaldi 5 years ago

satyajandhyala requested a review 5 years ago

satyajandhyala force pushed from cb569fe0 to abe982fe 5 years ago

SherlockNoMad commented on 2021-04-07

SherlockNoMad commented on 2021-04-08

SherlockNoMad commented on 2021-04-09

satyajandhyala force pushed from de4b27ff to 37e3dffb 5 years ago

satyajandhyala force pushed from 37e3dffb to 4f95ae06 5 years ago

yufenglee commented on 2021-04-09

SherlockNoMad commented on 2021-04-09

SherlockNoMad commented on 2021-04-10

SherlockNoMad requested changes on 2021-04-10

satyajandhyala force pushed from 4c9e2e36 to f0c02294 5 years ago

SherlockNoMad commented on 2021-04-12

satyajandhyala force pushed from f0b7c6f8 to 2d22f67f 5 years ago

ashbhandare commented on 2021-04-13

SherlockNoMad commented on 2021-04-13

weixingzhang commented on 2021-04-13

Added propagate_cast_ops option and PropagateCastOps transformation.

e9cf4e7f

Added InsertCastFP16Upstream

78b125ec

Added test cases to propagate Cast operations.

bd57e433

Expose GraphTransformerConfiguration to python interface and added pr…

1c1f9886

Added propagate_cast_ops option and PropagateCastOps transformation.

e9cf4e7f

Added InsertCastFP16Upstream

78b125ec

Added test cases to propagate Cast operations.

bd57e433

Added functionality to propagate Cast operations.

c1207859

Added RemoveBackToBackCasts.

eb52c17d

Added a new testcase.

71458710

Added code to add/remove edges when adding/removingCasts, in addition…

1fea1e53

Removed unused removed_nodes variable.

dddfaee9

Changed std::set to std::unordered_set

4d063d20

Added assertion.

32b3199f

Added RemoveUnnecessaryCasts function.

a98465de

Changed require_cast to a vector from a set.

5b9a67a4

Revert "Changed require_cast to a vector from a set."

a3bf24c1

Added logging.

64fc1fac

Check the type before inserting Cast.

316b146d

Perform Cast transformations in a loop until no more modifications.

baa76a5b

Added required_type_change argument SearchBackward function to collec…

a3e118f3

Stop backward search if a node had multiple outputs, such as Dropout.

1cf6dc65

Moved FuseSubgraphs to the beginning of the loop.

c62b3fe7

Resolve the graph at the top of the loop.

d6d477f5

Changed std::cout to VLOGS(logger, 1)

2dd815a8

Added ChangeTypeToFP16

24a0cbe8

Added PropagateFP32CastsFromInputsToOutput and PropagateFP16CastsFrom…

5749c43f

Modified SearchUpstream and SearchDownStream to return bool.

7907cc41

Changed propagate_cast_ops (bool) to propagate_cast_ops_level (int)

2511977d

Revert "Modified SearchUpstream and SearchDownStream to return bool."

6afe26c7

Added comments and TODOs.

42034f62

Check the number of outputs in RemoveBackToBackCasts.

9a2218a8

Fixed conflict resolution.

a63ca5d8

Apply cast propagation to the subgraphs.

4a84028f

Added graph_transformer_config and propagate_cast_ops_level option.

b76f9eed

Added removed_nodes to keep track of the removed nodes.

fbde5d35

Added more check for removed nodes.

e7b6350a

Zipped the test model paths and the expected output into a map.

434dbf24

Added different op codes to level 1 and 2.

ccb58c42

Added allow_list to add a list of allowed opcodes for cast propagation.

98f0d2d9

Disabled PropagateCastOp test because MatMul is not in the fp16 safe …

f9bf609d

Added propagate_cast_ops_allow to ORTTrainerOptions

8b28487d

Moved PropagateCastOps constructor to the source file.

ab158b00

Added BiasFastGelu and BiasGelu

40e04251

Changed allow_list from std::vector to std:unordered_set

99c1e983

Added a summary of the compute nodes changed.

d4864759

Added IsCastTo function.

c5b10773

Added IsType function.

1ccf3a92

Change GatherNames to ConcatNames

fa6cc0c0

Added a message a the beginning of each internal pass.

2c5e58d6

Fixed the frontend test.

2449452e

Fixed FuseNodes to create one output for Cast.

53ffb0dc

Added new tests cases for PropagateCastOps.

4dc47283

Formatting changes.

e0bb6dec

Don't remove back2back casts if the parent Cast is float->float16 and…

69cdc879

Changed the default value of propagate_cast_ops_level from 0 to -1.

b9841edc

Removed fp16_safe_ops collection and IsFP16Safe function and collecte…

d1bad31c

Iterate graph nodes using graph_view.

1fa2592f

Limit cast propagation to only cuda ep.

c16fa3d5

Added back commented testcases.

865b7b5d

Added summary

ee0683ff

Fixed compiler error/warning.

d9ba4a37

Fix RemoveCastsNodes call

8555f433

Added new test cases.

68056b98

Use Path to parse path and get the file name instead of GetLastCompon…

86217d10

Fixed Level 1 and Level 2, allow op lists.

3a930ce6

Removed scope for NodeIndex

cc4d87d8

Renamed RemoveCastNodes to RemoveCastNodesChain

4b71d977

Modified condition in side the for loop in ApplyImpl

488d658d

Removed compute_float16_... test cases as they don't result in transf…

8b13384f

Added more non-trivial testcase.

8f20637c

Added a TODO and modified nullptr checks.

720599ec

Added tests that use Add, MatMul and Transpose in allow list.

4ebd159d

satyajandhyala force pushed from 43057d67 to 4ebd159d 5 years ago

Added a log message printing the name of the nodes converted from flo…

477837db

Added more testcases.

39095614

Added more scenarios of using cast ops that require transformation.

2059b0bf

SherlockNoMad dismissed these changes on 2021-04-14

thiagocrepaldi requested changes on 2021-04-14

SearchDownstream should handle grapg outputs.

cbd52350

satyajandhyala dismissed their stale review via cbd52350 5 years ago

Added graph_transfomer specific options to the documentation.

71f9bc70

Fixed logging to list only the original node names in the converted n…

3c79b141

satyajandhyala force pushed from b0e44789 to 3c79b141 5 years ago

satyajandhyala requested a review from

thiagocrepaldi 5 years ago

thiagocrepaldi commented on 2021-04-14

Added docstings for the propagate cast ops.

eb9accf2

thiagocrepaldi dismissed these changes on 2021-04-15

typo

8034835c

satyajandhyala dismissed their stale review via 8034835c 5 years ago

thiagocrepaldi approved these changes on 2021-04-15

satyajandhyala merged 0da085ed into master 5 years ago

satyajandhyala deleted the ortmodule/propagate_cast_ops branch 5 years ago

onnxruntime
Propagate Cast operations to maximize lower precision (float16) computation
#7191

Merged

Propagate Cast operations to maximize lower precision (float16) computation #7191

onnxruntime Propagate Cast operations to maximize lower precision (float16) computation #7191 Merged

Propagate Cast operations to maximize lower precision (float16) computation #7191

onnxruntime
Propagate Cast operations to maximize lower precision (float16) computation
#7191

Merged