Where developers learn, build, and share. Your source for hands-on demos, cheat sheets, explainers and more.
Global
- Tweets 10,538
- Following 185
- Followers 65,844
- Likes 3,627
Inverted indexes are one of the core pillars of information retrieval.
It is literally THE thing that powers keyword (lexical) searches, by mapping terms against matching document IDs.
But, does every job and every data type require an inverted index? As it turns out, maybe not. Time-series data is a great example. Time series queries often actually ask "What are the values between two points in time?", rather than "What entries contain words XYZ?"
Elasticsearch leverages these properties, by only using inverted indexes where they add value.
TSDS uses doc values and skippers for relevant time-series fields instead of separate backing indexes. Text fields retain inverted indexes where full-text search is required.
Right tools for the right job.
1
1
4
1,372
We indexed 138M vectors in under 10 minutes.
Benchmarked on MS MARCO: 138M vectors at 1,024 dims, from ~4 TB of documents, on 8 NVIDIA RTX PRO 6000 GPUs at 95% target recall:
Built with NVIDIA cuVS @NVIDIAAI
- Indexing throughput: 38K docs/s on CPU, 281K on GPU
- 6x lower p90 search latency under concurrent indexing load
- Force merge: roughly 4 hours down to 5 minutes
- Same recall profile as CPU-built indexes
Benchmarks, setup instructions, and the Rally track parameters: go.es.io/3VPp1n2
1
4
4
1,529
Supabase monitoring in Elastic, with no agent to install.
Built in collaboration with @supabase
Available for Supabase Pro, Team & Enterprise on Elastic Cloud.
Integration: go.es.io/4rCe8B6
1
9
2,024
What is a doc-value skipper?
It's a small, but powerful way to efficiently filter values in Elasticsearch. It asks "does this block of doc values contain values in the required range?" by cataloging the minima and maxima for blocks of documents.
If the answer is no, the query can skip the block. If yes, dig into the block to check for matches.
This is a great solution for time-series data, because it is sorted by `_tsid` and `@timestamp`. The resulting blocks are composed of related values; and in turn, they are great for time and dimension filters.
You might ask - why do this? The main benefit is efficiency. Going from relevant inverted-index and BKD-tree indexes to doc value skippers reduced storage by 10 bytes/point in one benchmark.
It's one of the key components that power Elasticsearch's optimisations for metrics, and for Columnar Mode.
Try it out go.es.io/47iOrvK
2
2
5
2,092
Elasticsearch has made huge strides on its metrics store.
Metrics are unique. They are by nature ordered, being a function of time. So, Elasticsearch's TSDB uses these:
- Doc value skippers - fast scans & filtering through time range and dimensions without separate indexes
- Synthetic `_id` from `_tsid` and `@ timestamp`, and bloom filters support fast, efficient deduplication checks
- `_seq_no` trimming after safe merges, leveraging the fundamentally different usage pattern for metrics data
- ES95 codec further improves compression efficiency in Elasticsearch 9.5 and onwards
Benchmarks show each metrics point being reduced in size from ~25 to ~3 bytes per point, a huge reduction.
It might be a great time to consolidate your logs and metrics store to one place.
25 to 3 bytes per data point: go.es.io/4dPQawe
What's new in 9.5, including the ES95 codec: go.es.io/4iIY127
2
5
27
2,382
Your agent gave a wrong answer and you can't reproduce it.
Same prompt, different path. It picked a different tool this run, got an empty result back, and answered anyway.
Re-reading the prompt won't show you that. The trace will.
The OpenTelemetry GenAI semantic conventions provide vendor agnostic telemetry specification, where spans represent model calls and tool calls with arguments for messages attached.
Elastic reads the gen_ai.* attributes and puts token usage on the waterfall beside the spans. A slow run becomes a visibly expensive one.
Worth knowing: the conventions are still moving. Pin your semconv version and expect attribute renames.
Once tool calls are spans, an agent failure is a trace you read top to bottom.
LLM tracing in Elastic APM: go.es.io/4rk8Qd7
4
2
12
2,287
Your user's stack trace died with the browser tab.
They report that the app froze. You have no telemetry, because console.log() wrote to a console you will never see.
Server side you would never accept this. On the client side most teams still do.
The OpenTelemetry JS SDK treats browser logs as a first class signal: @opentelemetry/sdk-logs plus the OTLP HTTP log exporter, pointed at your collector. Elastic ingests the OTLP payload as it arrives, so there's no vendor format to maintain.
Before you ship messages:
- Redact sensitive data and PII such as emails and form values
- Sample telemetry, every user is now a log producer
Wire trace context through and the user's console output arrives attached to the request that broke, and correlates with server side signals. That's what actually shortens the investigation.
3
2
12
2,310
Images fill your context window before the question does.
A single 384px page becomes hundreds of visual tokens. Send
all of them to the LLM and there's no room left to reason.
jina-vlm sits an attention-pooling connector between a
SigLIP2-So400M/14-384 encoder and Qwen3-1.7B-Base. The
connector weighs which visual tokens carry signal, then hands
the text model a smaller set.
Two-stage training, multilingual by design.
Pooling costs you fine detail. Small type inside a dense table
is still hard.
Most people blame the LLM for bad answers on mixed documents.
The trade usually happens at the connector.
Read the paper: go.es.io/3TysG89
7
1
12
2,328
The HNSW vs DiskBBQ call comes down to recall and budget.
HNSW scales logarithmically and it is genuinely fast, but only while the vectors stay resident. Let them spill and latency climbs exponentially. Indexing pays the same tax, because building the graph means searching the graph.
DiskBBQ clusters vectors with hierarchical k-means and keeps them BBQ-quantized on disk, scoring only the clusters a query touches.
Need 99%+ and have the memory budget? Stay on HNSW. Cost-sensitive and fine at 95%?Go with DiskBBQ.
go.es.io/3UQa1Fh
5
5
19
2,347
An AI agent is optimizing Elasticsearch. Engineers validate on improved performance
Coding agents are trained on tasks judged by LLMs and humans. Being convincing gets rewarded. That's a problem when the question is "did this get faster?"
One agent, two phases with different permissions. Exploration goes broad on a real workload and ranks opportunities; a human approves the benchmark and promotes one into a scoped exploitation task. A statistical acceptance rule the agent can't bypass decides whether it lands, and a human reviews before the PR.
All validated with Elasticsearch engineers in the loop
7
5
90
1,896,182
jina-ocr-v1 is now available.
One model covers what otherwise takes a pipeline in OCR: layout, tables, math, handwriting, 100+ languages.
- Scores 83.4 on olmOCR-bench: highest of any OCR model under 600M active parameters
- Scores 91.14 on OmniDocBench, ahead of GPT-5.2 (86.59) and Qwen3-VL-235B (89.78)
- 3.4B total parameters, 570M active: the memory of a 3.4B model, the speed of a 570M one
- Input as PNG
- Output is Markdown, with HTML for tables and LaTeX for math
Also on the Jina API (10M free tokens) with downloadable weights on Hugging Face, CC BY-NC 4.0 licensed for research and noncommercial use.
Blog, technical report, and model card: go.es.io/4AkhUmv
Try it on your own scans via the Jina API, 10M tokens free: go.es.io/4yN0jSO
1
2
21
2,268
AlertZero: From Alert Noise to Attack Narratives: Relevance Please - Episode 9 nitter.net/i/broadcasts/1nxeLMmvz…
3
11
2,058
Cross-project search is GA on Elastic Cloud Serverless.
Search across serverless projects in different regions and cloud providers without moving data.
You can now link up to 100 projects by default (higher limits on request) from project settings in the Elastic Cloud console. Queries hit every linked project automatically. Route your search with project routing by region, cloud, alias, or custom tags. Data access control is consistent and complete wherever you’re querying from using Elastic Cloud API keys.
Works with Security, Observability, Search, and Vector Database project types, including serverless projects you are already running. Complete tier is required for Security and Observability projects.
Link the projects, open Discover, and run a cross-project query today.
11
8
61
1,704,909
Full GA details, setup walkthrough, and the per-feature nuances (alerting, SLOs, Synthetics, Streams): go.es.io/4yzE826
1
1
544
HNSW buys fast search by keeping every vector in RAM.
It needs them there to index too, because indexing means querying the graph you're still building.
DiskBBQ takes the other route:
- Hierarchical k-means groups similar vectors into small clusters
- BBQ compresses each vector 32x
- A query scores nearest centroids, then bulk scores only those clusters
On 1M vectors in memory: 94s to index against 1,054s. Recall 91% against 92%.
That's a point of recall for an order of magnitude on indexing. Above 99% recall the trade stops paying, and HNSW is still the answer.
17
8
89
2,952,670