We solved ML for vernacular Indian languages early on: DailyHunt

Virendra Gupta, co-founder and CEO of DailyHunt, bet on India’s diversity of languages to create a mobile app that publishes news, videos and content across 14 languages. Digital advertisers have taken to the managed technology platform. In 2018, he brought Umang Bedi as co-founder and president of DailyHunt. The company faces intense competition from big tech platforms like Facebook and YouTube. In this interview, the two leaders talk to Forbes India about the technology and strategy driving DailyHunt.

How have you created DailyHunt to be relevant to a diverse audience?

Virendra Gupta (VG): There are two aspects to India. One is the India which doesn’t reside in large cities, and the second is the India which doesn’t speak English. Nearly 70 percent of the Indian population speaks non-English languages. That was our product-market fit. That’s the India NewsHunt, which is now DailyHunt, was built for.

Our core thesis is India consumes content in vernacular languages. These users reside in the nook and corner of the country. In six years, we have reached 250 million monthly active users (MAU) who spend an average of 24 minutes per day on our platform.

Given that Big Tech platforms Google and Facebook dominate digital advertising, how does Dailyhunt hold its own in the competitive segment?

Umang Bedi (UB): It’s been about the trust and camaraderie Viru (Virendra) and I share that has laid the foundation on which DailyHunt has made some bold pivots. The first pivot we made when we came together in this journey was on the mission of the company. Our mission is to be the largest vernacular- or Indian language-focussed digital media platform that is helping consumers discover, consume and socialise with content which is not just informational but engaging and entertaining.

There were three big challenges in that journey. First was that the content ecosystem really exploded. We went from 1,000+ publishers to well over 50,000 content creators now. The second was building tech to ensure discoverability of that content—both on the backend to have the right signals, and a frontend change such that the user experience is more seamless with engaging and other entertaining formats.

The third challenge was monetisation—and the ability to control our own destiny because real world businesses are built on the foundation of profitable revenue and profitable growth. These are the three big opportunities we have been addressing.

The one big opportunity that we both see is if we look at the market around the world: in markets where the duopoly (Google and Facebook) operate, large multi-billion dollar media businesses have not been created.

Our belief is that there is an opportunity in India and it’s unique because it’s not just about one language. There are many Indias, so that’s the big goal we are trying to chase: create a large profitable multi-billion dollar enterprise around these multi-lingual users.

How do you ensure customer stickiness in a market with so many languages?

VG: In the end, the customer doesn’t care which product uses what tech. The customer-stickiness comes from finding relatable content. To do that across the depth and width of the country and 14 languages, we proliferated the content supply.

We went from 2,000 content creators to around 50,000 content-creators on the platform. Those 50,000 creators gave us content from different parts of the country in many genres—from news, local news, entertainment and health to sports and so on.

The second change we made was when we realised we had a lot of content but the user has short attention span. To deliver the best content in minimal time, we invested in a lot of machine learning and data science. We are one of the unique companies in the country that are working on personalising Indian content in local languages. There is a lot of hard work that has gone in to make that happen. User-stickiness has constantly increased from 14-15 minutes in 2017 to 24 minutes now.

UB: Personalisation is the ability to provide to the consumer what he or she wants to see in an instant on demand. That happens when you marry two variables-- a very deep understanding of content and understanding the consumer on non-personally identifiable variables like engagement—how they spend time, what are the signals of information they are passing on to you. Then, in real time, it is about having the artificial intelligence create a feed that is unique to our 250 million users today. That’s the science of delivering gratification to the user whose attention span is anyway extremely small.

How are advertisers taking to the digital advertising pie?

UB: The size of the advertising market is estimated to be $13 billion. Around $6.5 billion of that is spent on TV. Digital spends have interestingly reached $4.2 billion. The remaining is print and outdoor. Digital is growing at 30-35%, a much faster pace than the 10-12% at which TV is growing and lower single digits for print. In the next three years, digital is poised to be the dominant medium in India, as it is in any developed market.

DailyHunt monetises the educated Indian audience which resides in tier II and tier III cities and has a GDP per capita of well over $2,000—higher than the $1,800 GDP per capita we have in the rest of the country. It is not the long tail, but the belly of India. Charging higher really comes from understanding the user base. When we show them our knowledge of content and users, we are showing them the ability to target by location, language and demographics. Advertisers suddenly then have a very valuable audience in a target market that you could reach again and again.

The hardest thing to do in digital advertising is to unleash a performance-oriented business. Digital started off by being brand-only, but the largest platforms in the world have proved that real businesses are built on performance. That is something on which we have invested a lot of time—to build out an end-to-end performance-based business, leading to purchasing behaviour.

What kind of technology elements have made this possible for DailyHunt?

VG: One of the biggest competitive advantages that DailyHunt has is our tech stack, which includes engineering and data sciences. Today, our systems have been architected to run at a large scale and capture user signals in real time. Our whole engineering architecture is very scalable in a way that as customers grow, we can add servers without changing the entire architecture.

The biggest challenge is to make things simple. Today, DailyHunt gets 300,000 new content pieces every day. At any point of time, we have 2 million pieces of content of which we have to choose 10 pieces that can be recommended to a user. That’s a big challenge. To do that, our AI and data science investments in the past three years have yielded results. Our CTR has gone from 7% to 16%. (Clickthrough rate shows how well keywords and ads are performing. It is the number of clicks that an ad receives divided by the number of times the ad is shown).

This whole complex personalisation stack is built to understand the content, especially in local languages. When content comes, DailyHunt’s machine learning algorithms enrich it with relevant tags. This ML has trained over the years with 50 million customers. Once the content is enriched, we look at the customers’ attributes to check content performance in real time. This real-time loop gets fed back to the system to create a feed, which is highly personalised for the user. This is why I say DailyHunt has the best personalisation engine across languages in the country. Our DAU/MAU ratio (daily versus monthly visitors) has gone up from 28% to 38%.

UB: The scale of DailyHunt is such that the peaks become the new norm for us after every peak.

Can you explain how this happens?  

UB: Let’s take the national elections of 2019 – May 23^rd. At the time, we were above 200 million in MAU, with 40% DAU/MAU ratio. On that one day our DAUs more than doubled and the ratio doubled to 75-odd percent. So that is the first big peak that we saw. The second was the amount of consumption. In a single day, we beat the average amount of time spent on video in a month. The engagement around virality and sharing was five times the 50 million shares we do in a month.

Every single parameter of the system gets pressure-tested in a stress scenario. And those stresses are great for the business because, one, the entire tech backend that has been architected is in-house. Every single line of code is in-house. We run our own private cloud. Our load is tremendous—petabytes of data, terabytes of user profile data. DailyHunt also gives you a recommendation of content in real time by dynamically updating your feed based on your consumption. This takes a lot of science in the backend and server capacities. With more and more publishers, we add to relevant content, driving up time spent, and our monetisation also grows.

More content means more variety for the user. More time spent unleashes advertising inventory. More advertising means more revenue, a share of which flows back to the publishers. It is a virtuous cycle, with the peaks creating new norms. The system actually stabilises to a peak in a couple of months until we hit the next peak. That’s the way we think about the business.

What are the complexities that come with being multilingual?

VG: The complexities create business differentiation between us and the others. Each language has its own script, so rendering that font on a mobile phone itself was a big thing. We solved that pretty early in the game. Second, each language has its own nuances in terms of correlation of multiple words.

Typically, when people start doing data science and machine learning, they break a sentence into separate words and try to correlate them. But if you don’t understand the context of each word, you cannot understand the essence or emotion of that piece of content. There’s a huge amount of complexity over there which we have solved through supervised and unsupervised data learning.

The third thing is the relationship we have established in different parts of the country by plumbing the content ecosystem over a period of time. That is a great asset for us. We have people who have met publishers, and ensured we give money back to the ecosystem to make it a win-win situation for everybody.

How big is your technology team vis-à-vis overall headcount?

VG: We have 400 people in engineering and product. The overall headcount is 650.

What have the challenges been in accessing technology talent in such a competitive environment?

UB: When we go to any of the colleges we recruit from, we talk about two things that are unique to DailyHunt. In global organisations, it can be a couple of years before your line of code ever hits production. At DailyHunt you could be fresh out of school and have the opportunity to solve problems and deploy code over a very large infra.

If you really want to have impact, and drive a larger cause of serving Bharat, then this is the right place for you. It’s the right place for a problem solver. We are a young company. We have been able to attract young talent, which stays because we are mission-driven about serving Bharat.