• Resolved cloudres

    (@griotta)


    Hello,

    we started use your QUIC.Cloud CDN since yesterday – already read the FAQ also – but I didn’t understand exactly how the crawler works with the CDN.

    Our expectations were to use a CDN to serve our web site more quickly abroad to increase our positioning on Google. But it seems that your pops doesn’t cache automatically what the crawler scan.

    Normally with our main server without the CDN active it worked it this way and it was great, because all files were always “hit” by the users and also the Google crawler.

    Could you explain us better?

    Thanks a lot.

Viewing 15 replies - 1 through 15 (of 16 total)
  • Plugin Support qtwrk

    (@qtwrk)

    at current stage , the most optimal case/outcome for crawler is to crawl at your origin server , once cache is generated at origin , it will be served as static page to CDN node and then to end user , it is not efficient as cache at CDN node directly , but this way , all nodes can get the cache from origin

    Thread Starter cloudres

    (@griotta)

    Do you will working on it in the future?

    I’m asking that because with Cloudfront we had a ready preloaded cache on all nodes. It took to our web site some very good results (unfortunately then we had to abandon that solution due to the missing communication between LS and Cloudfront in the managing of the cache itself).

    Plugin Support qtwrk

    (@qtwrk)

    Hi,

    node cache/sync is on to-do list

    Best regards,

    Thread Starter cloudres

    (@griotta)

    Last question. Do you have a deadline for this? I’m asking you because we renewed right now a VPS to have the crawler on. But using Quic.Cloud the crawler presence became a little bit unuseful 😢

    Plugin Support qtwrk

    (@qtwrk)

    Hi,

    unfortunately I don’t really have timing for that

    a workaround is that if your targeted visitors are on certain specific region , you can make crawler to crawl on that node directly instead of origin

    on the other hand , if you want to use cloudfront for page cache , there are some options you need to disable to make it compatible with cloudfront : JS minify/combine , CSS minify/combine and webp replacement

    Best regards,

    Thread Starter cloudres

    (@griotta)

    I shared with our hosting manager your suggestion about the first workaround to use the crawler to cache the contents directly on your node. But they answered me that have not idea about what you’re talking for.

    Could you be more specific about how it can works?

    Thanks a lot @qtwrk!

    Plugin Support qtwrk

    (@qtwrk)

    okay , so let’s make an example

    in plugin -> general -> there is a “server IP ” setting , so let’s say your main visitors are in Spain , where 5.134.119.194 is the Madrid node IP , then you set this IP in the “server IP” setting

    once you start the crawler , it will directly use 5.134.119.194 as server IP to run , and since crawler was hitting on 5.134.119.194 which is CDN node , the cache is also generated in there.

    Thread Starter cloudres

    (@griotta)

    Wow, amazing. Easy! We’re going to try!

    The way to check if the preload cache is created or not is the same of always? Plugin -> Crawler, Hit/Miss. Correct?

    Thanks @qtwrk.

    Plugin Support qtwrk

    (@qtwrk)

    Hi,

    yes, it’s same -> crawler -> summary page

    Best regards,

    Thread Starter cloudres

    (@griotta)

    How can we find the IP of a specific node? In our case, we need the Milan IP.

    Plugin Support qtwrk

    (@qtwrk)

    Hi,

    well , the simplest way is (if yourself are located in Milan or nearby) , then from your computer visit the site , it should connect to Milan node and you can see IP there, or just simply ping your domain

    or , just go with 185.25.204.8 , it’s Italian node

    Best regards,

    Thread Starter cloudres

    (@griotta)

    Hi @qtwrk,

    could this trick cause some unexpected problems? Or is it a common procedure already tested? We have a shop online.

    Thanks.

    Plugin Support qtwrk

    (@qtwrk)

    Hi,

    no , worst case scenario , some people could come from other node which will be cache miss , that’s it

    Best regards,

    Thread Starter cloudres

    (@griotta)

    @qtwrk sorry for lot of questions, but it’s our work and it’s fundamental that if we will apply this edit it will works as expected.

    Actually our main VPS is near Milan, but obviously using Quic.Cloud Milan became the first node. More than our main node. Your last message disoriented me.

    We installed Quic.Cloud to speed up our user experience from abroad. So I need to understand exactly what happen if people from other nodes try to visit our web site that will be pre-loaded by our crawler on Milan instead that our VPS.

    In those cases the user will hit or miss these files?

    Thanks.

    Plugin Support qtwrk

    (@qtwrk)

    Hi,

    by aboard you mean like visitors come across all different regions ?

    in that case, it is better not to set server IP to node IP , but leave it to your origin server IP

    here is an example :

    Spanish visitor -> QC - Madrid node ->  your Milan server 
    
    Franch visitor -> QC - Paris node ->  your Milan server
    
    Italian visitor -> QC - Milan node ->  your Milan server

    so this is how the communication works , assume you run crawler on your origin server , then , after crawler has run , the situation will be like this :

    Spanish visitor -> QC - Madrid node / cache miss ->  your Milan server / cache hit
    
    Franch visitor -> QC - Paris node / cache miss ->  your Milan server / cache hit
    
    Italian visitor -> QC - Milan node / cache miss ->  your Milan server / cache hit

    and this is scenario if you set crwaler to run on Milan node

    Spanish visitor -> QC - Madrid node / cache miss ->  your Milan server / cache miss
    
    Franch visitor -> QC - Paris node / cache miss ->  your Milan server / cache miss
    
    Italian visitor -> QC - Milan node / cache hit

    this will speed up the visitors in Italy , but not as good as for other regions

    so it is more applicable if most of your visitors are focus on certain specific region , like what I have said in beginning

    a workaround is that if your targeted visitors are on certain specific region , you can make crawler to crawl on that node directly instead of origin

    if your visitors are not geo-specific , then better to crawler on origin

    once origin has cache , it will be served like static files when cache miss on nodes , which is already a lot faster than origin - cache miss case.

Viewing 15 replies - 1 through 15 (of 16 total)
  • The topic ‘Crawler efficacy with Quic.Cloud CDN’ is closed to new replies.