High-load data at global scale
The platform serves users from branded sites around the world. Every one of those interactions is also a data point that has to be captured, not dropped under load.
Case study · High-load data · B2B e-commerce
PriceSpider: where-to-buy data at hundreds of requests per second
A global where-to-buy provider needed solution architecture from scratch and a platform that holds up under high load on branded sites worldwide. Bintime built both, and the 23-person team that runs them.
- 23
- People in the dedicated team
- 11 yrs
- Partnership period, 2013–2024
- 2,000
- Brands and retailers served by the client
- 100s/s
- Requests per second the platform withstands
About the client
One of a handful of global where-to-buy providers
PriceSpider, formerly Hatch, has led the way in where-to-buy solutions and services since 2011. It works with leading global brands, retailers and agencies, helping boost sales across online and offline channels and building trust between brands and buyers by taking care of the customer journey.
In March 2015 the company received a €3.5 million investment from Vortex Capital Partners, which opened the way to double-digit growth and establishments in Amsterdam, Bangkok, Kyiv and Taipei. Its integration partners include Samsung, Lenovo, Braun, Canon, Microsoft, P&G, Walmart and Amazon.
- Industry
- E-commerce · B2B e-commerce
- Location
- Amsterdam, Netherlands
- Partnership period
- 2013 – 2024
- Team size
- 23 people, dedicated
Services
Expertise delivered
Technologies
Business challenge
Manage high-load data, and design the system that holds it
The project was considered a challenging one for two reasons: it required managing high-load data, and the solution architecture had to be created from scratch. There was no existing system to extend and no reference load to design against.
Architecture with no precedent
Creating the solution architecture from scratch meant the early technical decisions had to survive a decade of growth rather than the first release.
A team that did not exist yet
Delivering it required building an experienced technical team of 23 members — and relocating three teammates to the Netherlands to work closer to the client.
Approach
Engineer for the load first, then add the product
The development team helped PriceSpider cope with high-load data to withstand hundreds of requests per second. That is what made the client’s business goal reachable: serving users from branded sites around the world without the widget becoming the slowest thing on the page.
Delivery went out as an MVP and was extended feature by feature afterwards, so the architecture was proven under real traffic before the analytics and API surface were built on top of it.
Engagement modelA devoted team of 23 experienced professionals, three of them relocated to the Netherlands. Over the years the model evolved from outsourcing to outstaffing.
- 01
Streaming over batch. Kafka and RabbitMQ carry events; Apache Spark does the heavy processing, so ingestion is never blocked by analysis.
- 02
JVM where throughput matters. Java, Scala and Spring for the services on the hot path, Python where flexibility is worth more than raw speed.
- 03
Query where the query belongs. MongoDB for the document model, Elasticsearch for search and aggregation across gathered statistics.
- 04
Containerised delivery. Docker, Kubernetes and GitLab CI make scaling and deployment routine rather than an event.
- 05
MVP, then extension. Prove the core under load, then add the widget, the dashboards and the API — in that order.
Value delivered
Delivered as an MVP, then extended
A devoted team of 23
Experienced professionals assembled for the project, with three relocated to the Netherlands to sit closer to the client’s business.
Analytical widget for high-load statistics
The widget operates with high-load statistical data gathering — the piece that runs on brand sites worldwide and still records what happened.
Conversion monitoring and reporting dashboards
Extensive reporting gives brands insight into customer behaviour, so where-to-buy stops being a black box between click and purchase.
A fully customisable API
Partners integrate on their own terms rather than around a fixed contract — which matters when the partner list includes Amazon and Walmart.
From outsourcing to outstaffing
The engagement model changed as trust grew: the same people, now working as part of the client’s own organisation.
Expanding cooperation
Where the partnership stands now
The Bintime team continues maintaining the solution that brought PriceSpider to where it is now — one of a handful of global where-to-buy providers.
PriceSpider leads the space through its pace of innovation: adding physical retailers to its global network, developing shoppable reviews, and offering tools for digital shelf management.
FAQ
Questions we get about high-load platforms
What is a where-to-buy solution?
A service that shows shoppers on a brand’s own site where the product can be bought, online and offline. PriceSpider has led that space since 2011.
What does high-load mean here?
Hundreds of requests per second from branded sites around the world, with statistics gathered from each interaction rather than sampled.
Which technologies run the platform?
Java, Scala and Spring with Python services, RabbitMQ and Kafka, Apache Spark, Splash, MongoDB and Elasticsearch, on Docker, Kubernetes and GitLab CI.
How did the engagement change over eleven years?
It started as outsourcing and became outstaffing: a devoted team of 23, three of them relocated to the Netherlands, working as part of PriceSpider.
Was the platform delivered in one release?
No. An MVP first, then the analytical widget, reporting dashboards and the customisable API once the core was proven under load.
Do you relocate engineers to the client?
Where it helps. Three teammates moved to the Netherlands on this project to work directly alongside the client’s business.
Start a conversation
Data volume outgrowing your architecture?
Tell us what your traffic and data look like today. We come back with an architecture direction, a team shape and the first ninety days.
- Reply within 24 hours from someone who can scope it
- Reference calls with existing clients, where they agree
- NDA signed before any detail is shared
Thank you
We will get back to you within 24 hours with next steps.