of Shopify, recently told his employees in an internal memo: “Before asking for more headcount and resources, teams must demonstrate why they cannot get what they want done using AI”.
Having worked in startups for the past 6 years, asking for more headcount or more resources is usually not an option anyways. Constraints are tight and you generally have to rigorously invest in projects you are confident will be impactful. So in these situations, Tobi would probably rephrase: “Suck it up and just use AI if you can”.
As a Data Scientist, I want to understand how our work is evolving with AI. Tech Executives are clearly expecting every team to be more efficient and more creative. But can a multi-billion parameter model, although it has read the entire Internet, be systematically helpful at solving your own problems? To tackle this question, I am proposing the following framework: let me go through all the projects I have worked on since the beginning of my career and assess how much AI would have helped.
Today, we go back to 2020. I am a junior Data Scientist at a company that has been hit pretty bad by the pandemic: Rent the Runway.
What the Project was about
Rent the Runway was launched in 2009. The company experienced rapid growth from 2016 to 2020, after introducing their most popular product: a monthly “unlimited” subscription to fashion, aka “Closet in the Cloud”, allowing you to rent a huge number of high end clothes at an unbeatable price. The product was a hit for every woman wanting to regularly wear something new at work, night outs, parties, special events etc. So obviously, when Covid started in March 2020, and everybody stopped going out for weeks… well, it kinda killed the vibe.
The “Netflix of fashion” (yes, some people really used that nickname) ended up with an insane amount of unused inventory, an entire season of items that will just have to “sit” in a warehouse, and of course a huge revenue decrease. It was urgent to find a new revenue stream to survive financially. Not the right time to ask for more resources or headcount, as a third of the workforce was furloughed.
Here came a brilliant idea: what if we were trying to come back to the retail business? That is, selling items as second hand instead of renting them. But here was the big question: as the lockout is going to end one day and people are going to go back to renting, what items should we keep for now vs. sell for a discount? And how much should this discount be?
The 2020 Solution
The goal of the project is to get for each product the optimal price, that will be the right balance between renting and selling. You can get the optimal price p as the price that will maximize the following:
Which is easy to find… assuming you know the future rental revenue (the “RentalRev” in this equation) and the price elasticity (the probabilities in this equation).
In early 2020, I was already working on RTR unit economics and revenue forecasting. I was building a model to predict, based on an item rental history, how many more times it could be rented and what additional revenue it would generate.
The missing piece was having an idea of pricing elasticity, i.e answering the question: given a price for an item, what would be the probability of selling it? To know more about this model, I would redirect you to this very detailed and well-written blog article by my teammate Meghan Solari.
It is important to note that some business constraints had to be applied to make sure that we would not sell off an entire style and keep some units for rentals.
How AI could have helped
This project is close to a classic demand and supply problem, with the twist of the rental vs retail revenue that makes it a bit more interesting. But finding the equation that gives the optimal price is not the main challenge. The main challenge is how to estimate each parameter given insufficient data.
Indeed, predicting future demand is hard: you only have a few months of history (at best) for each style, and you need to predict a large horizon (basically up to end of life). Rapid changes in fashion trends require a deep understanding of the industry to be predicted, if predictable at all. And the uncertainty created by the early Covid period made any time series models very hard to build.
Estimating pricing elasticity is not any easier. As Rent the Runway was not a retail business, sales data was by design limited.
And that’s exactly where the challenge would come for any AI-driven solution as well. An AI can only be as good as the data it is being provided.
Solving for the sparse style-level data
Even though each style has limited history, there’s a wealth of information in similar items. This is a prime use case for transfer learning and shared embeddings that could have been made easier by the access to pre-trained LLMs. Shared style-level embeddings could have allowed us to make strong assumptions on new styles based on metadata: color, brand, price, fabric, silhouette… We could have more effectively built models that learn how to predict demand curves from a few data points, drawing from patterns in historically similar items. A company like Stitch Fix has been pioneering this space by using item metadata to create deep embeddings that generalize across new inventory.
Keeping up with Fast fashion cycle
LLMs could have made it easier to follow and understand ever-changing fashion trends and work on external signals to predict potential shifts in the entire industry. That was not something that was easy in 2020, because it requires scrapping massive amounts of data, finding out what is relevant and interpreting weak signals. Today, that is exactly what LLMs are good at. Companies like Trendalytics do just that, scanning TikTok, Google Trends, and social media to surface emerging patterns in silhouettes, colors, or influencers’ posts. That data would have been extremely valuable to make an accurate demand forecast.
Building a dynamic pricing Agent
One last thing that could have been fun to explore, given today’s technology, is to build an agent that would have modified the prices in real time and learnt, through reinforcement learning, the optimal pricing strategies by interacting with the environment. That could have allowed us to make sure the prices depend on the style’s historical and future demand but also on the customer features, i.e personal rental and purchase history, engagement, taste, etc. That could have brought us closer to what top RL teams at Airbnb or Uber do, continuously adjusting prices based on real time demand and booking probability.
These are some of the ideas that I selfishly would have been super excited to work on, but note two important things:
1. From a product perspective, it is really hard to estimate (especially now that I don’t have access to the data anymore) what the impact on overall revenue would have been.
2. These ideas could have also been built in-house back in 2020, given the great team of ML engineers we had at Rent the Runway. But it would have represented months — if not years — of research and development with high risks, which we could not afford at that time.
And that is probably my main takeaway so far on LLMs: they do not trivialize the problems we used to bang our heads at 5 years ago (or not yet) but they make it easier to test ideas that would have taken an unrealistically long time to develop back in the days. This changes the paradigm in which Data teams typically operate and opens new opportunities of partnership with Product teams.
Article 100% human written
