[metadata] add option of configuring ua of async metadata (#74594)
### What
Introduce an experimental config `experimental.htmlLimitedBots` (type
`Regex`) which can used for controlling the metadata streaming behavior
for different bots, if the bots user agent is matching the regex, it
will serve blocking metadata forcedly. This gives users some control to
opt-out streaming metadata for some clients or bots which cannot handle
it.
Output another manifest `response-config-manfiest.json` containing the
customized UA, which is potentially used for the platform serving
Next.js to control the returned response for certain UA. But it's mostly
for PPR case which we'll tackle later.
Closes NDX-635
Closes NDX-599