Are we able to send AT commands like PSM or eDRX setting changes over the terminal window in nrfCloud?
If yes, please provide sample json request.

Hello,
Have a look at the nRF Cloud Github repo. Under Application-Protocols/Schemas/CloudToDevice there are several examples on how to send JSON messages from Cloud To Device, among others modem:
{
"appId":"MODEM",
"messageType":"CMD",
"data":"AT+CFUN?\r\n"
}
I have not tested with PSM, you will need add this yourself.
Kind regards,
Øyvind