Two upload folders – one private and open to indexing and downloads?

  • Resolved SunnySunflowers

    (@sunnysunflowers)


    11 years, 6 months ago

    PROBLEM:
    1) I have pdf files I want to provide in a link to users.
    – I don’t want these to be directly downloadable.
    – I don’t want the folder and the files indexed in a search engine.

    2) I have other uploaded files that I want a user to be able to download directly. It’s fine with me if they share the link or the file. I also want these indexed by search engines.

    PARTIAL SOLUTION:
    – I found a plug-in called “Email Before Download” that will provide a masked link to the file. From my testing so far, this will take care of the masking issue without adding too much overhead to the end user process.
    – The problem is that even if I give a masked link, search engines will still index the upload folder/files. So the files will be out there anyway.

    Is there a problem with having two upload folders? One for unrestricted access files and another for the restricted files?

    I guess I will have to use ftp (WinSCP) to get the files into one of the folders. But how can I prevent the restricted folder/files from being indexed by search engines? Will adding an .htaccess prevent indexing, and, if so, what would be the code.

    I am very new to all this, so assume I know nothing.

Viewing 6 replies - 1 through 6 (of 6 total)
  • Krishna

    (@1nexus)

    11 years, 6 months ago

    You can upload to a sub-directory, link to the file in your posts/pages and disallow that directory from crawling and indexing by robots.txt file.

    Pioneer Web Design

    (@swansonphotos)

    11 years, 6 months ago

    and disallow that directory from crawling and indexing by robots.txt file.

    To do this properly, do use a robots.txt file, but also use tools at the search engines to stop listing of these url’s – robots.txt is respected by most bots (when it comes to crawling, which is not the same as indexing or listing) but not all, asking to not list first is the key.

    Order: Crawl | Index | List

    Bots can and will crawl all links, files, dir’s it can Read…It is up to you to note what to be listed…

    Thread Starter SunnySunflowers

    (@sunnysunflowers)

    11 years, 6 months ago

    Thanks Krishna and Seacoast for the tip about robots.txt.

    Seacoast, what tools are there to stop listing of the URLs?

    UPDATE:
    I’m going to answer (maybe) my own question. I found a plug-in, WP Robots Txt, that may be the kind of tool to which you were referring.

    Pioneer Web Design

    (@swansonphotos)

    11 years, 6 months ago

    Seacoast, what tools are there to stop listing of the URLs?

    No, I was referring to the tools available at the major Search Engines.

    Thread Starter SunnySunflowers

    (@sunnysunflowers)

    11 years, 6 months ago

    Thanks Seacoast. Google web master tools look like they would be good for this.

    Pioneer Web Design

    (@swansonphotos)

    11 years, 6 months ago

    And as all follow Google, the rest will follow…

Viewing 6 replies - 1 through 6 (of 6 total)
  • The topic ‘Two upload folders – one private and open to indexing and downloads?’ is closed to new replies.