understand and plan an elasticsearch cluster

Vote count:

0

I'm sending per minute hundreds of async requests to http bulk api, each request with 10 to 1000 doc_as_upsert operations.

First attempts I was indexing user-based and the writes were extremely fast without any overhead.

So now I'm trying to index date-based to improve search performance, but write speed is very slow. I'm using data of 4 years, so I'm creating many many daily indexes per request (eg: "content-2014-17-04").

I'm testing in Ubuntu 12 droplet with 2gb RAM and two CPU's.

With HTOP I see RAM usage stable in 50% and one of two CPU's are every second hitting 100%.

I'm using routing to keep data nearly too.

1. What is causing this slowness?


2. Adding more shards can be a good option on this?


3. What is better? more machines with low CPU's and RAM, or less machines with more CPU's and RAM?

asked 37 secs ago

user3175226

176