Support » Plugin: Stream » Scaling with tons of data – Performance

Viewing 5 replies - 1 through 5 (of 5 total)
  • Hey Scott, we use Elasticsearch, so it’s pretty scalable. Right now we have over 30 million records and it’s humming right a long.

    That being said, we do have a fair usage asterisk that for sites that have consistently over 100k records per month they should probably use a custom enterprise setup. Especially for a private multisite network, because there are cool things you can do with a custom setup that you can’t do with our hosted platform, like view all activity network-wide.

    I have a meeting with your friend Taylor Lovett about this tomorrow, actually 🙂

    Sorry I didn’t answer your performance question very specifically.

    Here were our average API response times this week, and like I said our cluster has 30m+ records:

    Create a new record – 21ms
    Search existing records – 25ms

    Thread Starter Scott Kingsley Clark

    (@sc0ttkclark)

    I’ve got a meeting with Taylor about something else tomorrow myself 🙂

    Thanks for the quick answer. Do you or are you planning on offering the custom setups or integrating with a different Elasticsearch instance?

    Both. We can host/manage a private Elasticsearch cluster or facilitate for clients to use their own on-premise cluster that they would manage. All of this comes in the form of a separately licensed add-on plugin we have that interfaces with the main Stream plugin.

    Thread Starter Scott Kingsley Clark

    (@sc0ttkclark)

    Gotcha, that answers my questions then 🙂

    Thx!

Viewing 5 replies - 1 through 5 (of 5 total)
  • The topic ‘Scaling with tons of data – Performance’ is closed to new replies.