transformers

ec532f20 - feature: Add robust token counting with padding exclusion (#40416)

feature: Add robust token counting with padding exclusion (#40416) * created robust token counting by using existing include_num_input_tokens_seen variable and kept bool for backward compatibility and added string also to ensure everything goes well and kept default as is. also robust test cases are created * some codebase mismatched in my local and remote, commiting to solve it and also solved code quality issue * ci: retrigger tests * another attemp to trigger CI for checks