Hello! Here's how I do it on a phone. A computer may be different. Ignore the ads popping up on some of my screenshots.
"Attach files" button at the bottom of the text box, which you are aware of:
Once you choose your photo, wait for it to load all the way.
The photo is now fully loaded and visible and you can click/poke "Insert" in the photo's top left corner. I always do "Full Image" so it's bigger.
The image is now nested in your post, wherever your cursor was.
That last insertion step is technically not 100% necessary to Attach a photo to a post. It will still show up as a clickable thumbnail at the bottom of the post once published (I think?), but people may not see it, so it's better to insert it in the post which is what most people do.
I have had an issue lately that may just be my internet, but sometimes a photo I'm trying to attach won't upload all the way and will be gray instead of a thumbnail at the bottom. When that happens, I am unable to click on it to insert it. I just delete that attachment and try again, sometimes several times. I'm not totally sure why that happens, I will have several load just fine and then one I have to fiddle with.
I hope that helps!