Exclude files from wordpress for hotlinking

This has probably been covered, but I’m not sure what it’s called so I can’t find it.

WordPress is installed in the root of my site, with the multiblog option for subdomains (if that matters) However, sometimes I want to use my server for general file storage, hotlinking images or audio, etc. How do I get those folders to exclude from wordpress? Example, I know that mysite.com/images/image.png exists because I uploaded it via FTP, but when I go to that url, I get my wordpress “page not found” page.

I know there’s a way to do this because I’ve done it before… by simply uploading the files and linking the paths. I’m not sure why it’s not working anymore.