It's been a few years so the details are fuzzy, but iirc it was just simple things like index sizing, managing shard count, some JVM tuning, certain aggregated fields blowing up once we had more data in our instances...
We also got our data very out of order. We had embedded devices logging analytics that would phone home very infrequently, think months between check-ins
I forget why but that became a big issue at some point, bringing the instance to its knees when a few devices started to phone-in covering large periods of time.
ES just has a ton of knobs, I imagine if its been important to you, you have people specializing in keeping it running, which is great... but the amount of complexity there that is specific to ES is really high.
It's not like there's no such thing as a Postgres expert for example, but you don't need to hire a Postgres wizard until you're pretty far in the weeds. But I feel like you should have an ES wizard to use ES, which is a little unfortunate
We also got our data very out of order. We had embedded devices logging analytics that would phone home very infrequently, think months between check-ins
I forget why but that became a big issue at some point, bringing the instance to its knees when a few devices started to phone-in covering large periods of time.
ES just has a ton of knobs, I imagine if its been important to you, you have people specializing in keeping it running, which is great... but the amount of complexity there that is specific to ES is really high.
It's not like there's no such thing as a Postgres expert for example, but you don't need to hire a Postgres wizard until you're pretty far in the weeds. But I feel like you should have an ES wizard to use ES, which is a little unfortunate