Anatomy and mindset of the data army at Gojek

Stories from the land of data at Gojek

gojek.jobs

At Gojek, we build products that help millions of Indonesians commute, shop, eat and pay, daily. The Data Engineering team is responsible to create a reliable data infrastructure across all of Gojek’s 18+ products.

Mission

We aim to provide disproportionately large advantages to Gojek over its competitors by making data available, accessible, reliable and actionable at scale.

The data platform allows internal teams to build on our products to develop innovative solutions; creating new opportunities for our customers and expanding on what’s possible.

Approach

Some informal philosophies and approaches we follow during our time working on data infrastructure:

Scale

Data at Gojek doesn’t grow linearly with the business, but exponentially, as people start building new products and logging new activities on top of the growth of the business. We currently see 6+ Billion events daily and rising.

Automation

Working at such a large scale makes it really important to automate everything from deployment to infrastructure. This way we can push features faster without causing chaos and disruption to the production environment.

Product mindset

We operate as an internal product organization. We measure success with business metrics like user adoption, retention, and the revenue, or cost savings generated per feature. Our customers are Product Managers, Developers, Data Scientists, and Analysts.

• Asgard: A case study to envision data infrastructure automation at GO-JEK

• Chronicle: A story of building technical documentation for GO-DATA

Anatomy

We are a team of full-stack, opinionated, polyglot engineers and passionate human beings who believe the core of a developer’s job is not just writing code.

We own the full product cycle and are equally responsible for adoption and experience as our product manager. Having product ownership gives everyone a responsibility to put their best and gives meaning to our daily work.

• The art of being full stack

• Scaling culture at GO-JEK

Work

As data engineers, we solve problems with data to improve the product that we offer to our users.

We build tools, infrastructure, frameworks, and services and on the way develop new skills, new ways of doing things, new tools, and turn our backs to traditional methods. At GO-JEK, data engineering is much closer to software engineering and we understand the tradeoffs between different open source Big Data technologies and aren’t afraid to build it ourselves if needed.

• ATLAS: GO-JEK’s real-time geospatial visualization platform

• Loki: Chaos engineering tool for data infrastructure at GoJek

As a member of the data team, you’ll have immense impact — shaping the data with which the company makes innumerable business decisions.

Tools

Given the real-time nature of the business at GO-JEK, the entire data infrastructure from the ground up focuses on real-time data processing, unlike traditional batch processing architectures. Follow the below post for a simplified view of our data architecture and tools.

• Data infrastructure at GO-JEK

• How our DIY platform creates value through network effects

• Olympus: Terraforming repeatable and extensible Infrastructure at GO-JEK

We also rely on open-source and are thankful to the collaborative data engineering community for inventing new data technologies and open-source them. Our work wouldn’t be possible without the foundational work of so many engineering teams before us.

If you like what you’re reading and interested in building large-scale infrastructure that excites you, do check out our engineering openings at gojek.jobs. As always, I would love to hear what you guys think.

---

Anatomy and mindset of the data army at Gojek was originally published in Gojek Product + Tech on Medium, where people are continuing the conversation by highlighting and responding to this story.