Fix robots.txt generation (#47625)
### What?
Fix robots.txt generation
### Why?
Should support multiple allow/disallow rules, they should be generated in different lines
### How?
Closes NEXT-905
Fixes #47224
fix NEXT-905 ([link](https://linear.app/vercel/issue/NEXT-905))
