+ 'shift' (delete without moving to recycle), then everything worked as expected. Symptoms: standard (nonadmin) user wanted to delete a file (move to Recycle bin ) with 'delete' key on keyboard or 'delete' option in context menu. I finnaly resolved my issue and I would like to share it with You all.
Giving the headache, just like in this other thread still being unaswered:
The real issue with UPD permissions is still Dharmesh brings the 'temporary profile' issue where the proper UPD (VHDX file) is not mounted, so the temporary one is given by the RDS.
The question is about permissions to the files in users UPD (User Profile Disk). This 'answer' shouldn't be marked as answer because it doesnt't really relate to the question.