Bots + Exchanges: A Perfect Storm

The symbiotic relationship between crypto trading bots and exchanges

Trading bots play a vital role in making the trillion-dollar crypto exchange industry more liquid and efficient. Both centralized and decentralized exchanges need bots, but for different reasons.

To explain this dynamic, I gave the following talk at a couple of conferences (Avalanche Summit, ETHPortland) recently and will do an updated version at EthCC later this summer.

Below is a lightly-edited transcript.

---

When my startup pivoted to building an open source crypto trading bot 3 years ago, everyone (but especially our VCs) were like: What? Why? And most importantly, how will you make money?

At the time, to be honest, we didn’t really know either. But the idea seemed to resonate with exchanges, so after getting a small dev grant from 0x and striking a partnership with Binance, we decided to go for it.

Today, Hummingbot is the industry’s leading open source trading bot, used by tens of thousands of traders and developers all over the world. We are now a globally distributed team spanning 20+ countries, split between a for-profit company and open source foundation.

Along the way, we discovered that trading bots and exchanges have an interesting symbiotic relationship that not many people understand, so I want to explain that to you today.

First, a couple of definitions…

What is an Exchange?

I often use the analogy of a bazaar (a flea market to us uncultured Americans) to explain how a crypto exchange works.

A good bazaar has a diverse selection of products for sale, but what makes that happen are the individual merchants. If there are only a few merchants, shoppers wouldn’t come to the bazaar. And if there are only a few shoppers, merchants wouldn’t want to set up their stalls at that bazaar. A bazaar is a two-sided marketplace that needs both merchants and shoppers to function properly.

Crypto exchanges are similar. Traders arrive at the exchange to buy and sell tokens, but their ability to trade with low transaction costs depends on others who provide liquidity. Without these liquidity providers, traders wouldn’t want to come, and without traders, it doesn’t make sense for the liquidity providers to show up either.

In exchange jargon, we call the liquidity providers the market makers (or just makers) and the traders the takers.

This is a colorized photograph of Exchange Place in Manhattan, around 1900. Not too dissimilar from a bazaar, right?

The people standing on the street are market makers, who each ran individual businesses buying and selling stocks to passerby. Eventually, they decided to join forces and operate as a collective in a nearby building, which became the New York Stock Exchange.

Naturally, these makers didn’t like competitors, so they tightly restricted access to liquidity provision. Even to this day, you have to be a Designated Market Maker to provide liquidity to the NYSE, an exclusive role available only to the largest trading firms.

But then along came cryptocurrencies, and all of a sudden, anyone who could create a website or write a smart contract could launch a crypto exchange.

Because of that openness and the resulting competitive dynamics, the crypto exchange industry has evolved in a radically different way from what exists in traditional finance.

Just like bazaars, crypto exchanges are pure marketplaces, so they had to cater to both makers and takers by offering free, open API access to everyone. NYSE and other traditional exchanges charge millions of dollars per year for this direct market access.

It also enabled a lot of experimentation and creativity. Today, there are thousands of crypto exchanges of all stripes and styles, trading both spot and derivative markets, both centralized (database-based) venues as well as decentralized exchanges that live on myriad blockchains.

I hope that gives you a broad overview of exchanges, so let’s talk about bots.

What is a Bot?

Many people think of trading bots as malicious, AI-driven agents, but having seen how users really run bots over the past 3 years, I use a much more basic, boring definition:

A bot is a process, a computer program that loops continuously. In each loop, a bot fetches data from one or more exchanges, performs some internal logic that tells it how to proceed, and then attempts to execute transactions on those exchanges. Like any computer process, a bot keeps running until it’s terminated, either by the bot-runner or by some external event.

In fact, a key bot metric is how long they can run before being terminated by some external event outside the runner’s control, bugs in either the bot code or the exchange code, WebSocket stream disconnection, memory or storage issues in the cloud instance where the bot is deployed, and even node infrastructure issues in the case of decentralized exchange bots.

Identifying and fixing these edge cases can be tricky, which is why a good QA testing process is important in building a good bot.

Bots are typically clients, not protocols. A client is software that you run on a machine where you have exclusive, secure access. That’s because in order to continually send transactions, the bot needs unencrypted access to your API keys for centralized exchanges like Binance, as well as wallet private keys in case of decentralized exchanges like Uniswap.

Therefore, most bots are deployed on cloud instances like AWS, ideally in the same geographic region as the exchange’s cloud servers to minimize latency. While there do exist web-based platforms where you can input your API keys and run bots, they are generally aimed at retail traders. As the saying goes, “Not your keys, not your crypto.”

Despite the popular misconception that bot-runners are evil hackers, the reality is that most bots are run by individual traders and developers just trying to make side income. 

Some traders make enough money to hire their friends, and that’s basically how all quant trading firms start off, even the largest firms like Jump and Citadel.

How Bots Interact with Exchanges

Most exchanges in traditional finance, as well as most centralized crypto exchanges, use an order book model. Makers submit limit orders that specify how much of a specific token and  what price they are willing to buy or sell, and the order book aggregates all of those orders in one place. Makers can cancel or adjust their orders at any time, typically for free.

The aggregate order book provides the liquidity that is available for takers, the traders coming to the exchange looking to buy and sell tokens, to consume.

Because of market price volatility and competitive dynamics, makers need to rapidly create, adjust, and cancel orders in order to not lose money. While human-powered “click farms” still exist, most serious makers run market making bots to automate this activity.

Above is a diagram of a simple market making bot. Each loop, it fetches the state of the exchange order book and decides if it needs to refresh the orders it has placed.

Let’s say a market making bot has placed a buy order 1% below the current mid price, the current price in an order book, defined by the average of the best buy and sell orders. It has also placed a sell order 1% above the mid price (makers call this making a two-way market). 

If the bot fetches the order book and discovers that the mid price has moved substantially in either direction, the bot will likely need to refresh its orders in order to continue bracketing the mid price.

But order books aren’t the only exchange model in crypto. Automatic Market Maker (AMM) decentralized exchanges, popularized by Uniswap, exploded in popularity in 2020. Today, they account for billions in daily trading volume and are the fastest growing sector of the crypto exchange industry.

AMMs enable makers to provide liquidity simply by depositing two tokens into a pool. The smart contract has simple internal logic that sets a price for any given quantity that a taker want to buy or sell, with the condition that the product of the quantities of the two tokens in the pool must be the same before and after the trade (x*y=k). Because an AMM allow makers to provide liquidity in a single transaction whereas they would have to constantly refresh orders in the order book model, AMMs are a perfect fit for blockchains, where every transaction costs gas.

But there’s a catch. Since an AMM is just a pool of two tokens and all makers can do is to add or remove liquidity, the only way that AMM’s mid price can change is via trading. That means that whenever market prices shift, arbitrage opportunities are created. While some trades are done by retail traders coming to swap tokens, the bulk of AMM transactions are executed by arbitrage bots.

Above is a diagram of a simple arbitrage bot. Each loop, it fetches prices from two different exchanges that trade the same trading pair. 

If the two markets exhibit a meaningful price difference, the bot will execute a buy transaction on the lower-price exchange and a sell transaction on the higher-price exchange, locking in an arbitrage profit. If there’s no arbitrage opportunity available, nothing happens and the bot loops again.

So the status quo in the crypto exchange industry today is that retail traders are takers consuming liquidity, on order book exchanges, while they are makers providing liquidity on AMMs.

Conversely, market making bots are the makers on order book exchanges, while arbitrage bots are the takers on AMMs.

The industry has evolved this way because retail traders gravitate toward user-friendly, simpler styles of trading, like earning LP yields or taking from an order book, but to sustain that type of experience for them, the marketplace need some type of automated agent (bot) to provide the necessary liquidity or volume.

In trading parlance, a functioning market matches toxic flow (professional bots) with non-toxic flow (retail traders).

But since bots are programs that can do anything, they aren’t limited to just arbitrage and market making.

• Hedging bots allow you to protect against downside risk by hedging on another market. For example, our Cross-Exchange Market Making bot combines market making and arbitrage to provide liquidity to one exchange while hedging filled orders on another exchanges.

• Smart order routing bots improve the execution of a trade, by splitting up a large block into smaller chunks that are routed to different exchanges, or dripped onto an order book over time. One example is Hummingbot’s TWAP strategy.

• Technical indicator-based bots execute trades based on a technical indicator like RSI or MACD. For example, the Aroon Oscillator strategy in Hummingbot computes the Aroon indicator based on order book data and uses it to improve Hummingbot’s base market making strategy.

That’s why we designed Hummingbot as an open source framework. You can run the off-the-shelf strategies, or you can build your own custom bots.

The possibilities are endless.

The Hummingbot story

Let me tell you more about Hummingbot.

Before we launched the open source project 3 years ago, what sparked the idea was meeting lots of people trying to build trading bots: DEX startups, market makers, and even individual traders.

They were all spending weeks doing the exact same thing: building exchange integrations that let them do high-frequency trading. This entailed debugging low-level WebSocket connections, and wrestling with poorly-documented exchange APIs.

Even worse, they had to do all this drudgerous, mind-numbing work for every single exchange before they could run a single trading bot. When you are sending 10 orders a second, since any bug, however slight, could cost you a lot of money.

In hindsight, the answer seemed obvious: an open source project with integrations to every exchange, each built and maintained by community members, that everyone in the world could use.

Not only does Hummingbot maps different exchange APIs into a single, unified standard, but it also offers integrations to both centralized and decentralized exchanges.

Currently, the two worlds are highly fragmented for the simple reason that the two sides don’t speak the same language from a technical standpoint:

• Code: CEX traders are Python/C++ shops while interacting with DEXs requires familiarity with Solidity/Cargo and/or the ability to consume Typescript SDKs.

• Matching model: CEXs typically use the central limit order book (CLOB) model to match buyers and sellers, while DEXs are typically AMMs.

• Architecture: Connecting to CEX APIs requires familiarity with REST/WebSocket APIs while connecting to DEXs requires familiarity node/wallet/smart contract infrastructure across various blockchains.

With Hummingbot, quants can build cross-chain, cross-exchange bot that captures arbitrage and liquidity provisioning opportunities between any two exchanges in the network, say between Binance and Uniswap, or Coinbase and dYdX,

But Hummingbot isn’t just an open source codebase.

We have the largest and most active community of quant traders in the crypto industry. Thousands of people have learned market making through our YouTube videos and getting support on Discord.

Since Hummingbot is client software rather than a blockchain protocol, we happily welcome forks. Over 1500 developers have forked Hummingbot for their own use since we started.

Finally, we are starting to see significant institutional usage. Many token issuers and exchanges use Hummingbot as an alternative to hiring market makers, and new market makers now build their trading stack on top of Hummingbot to skip months of development work integrating to exchanges.

As the Hummingbot codebase matures and our community grows, I believe that it will eventually let users run any type of high-frequency trading strategy across any set of exchanges, anywhere in the world.

This is our mission: open source HFT.

Of course, this would be incredibly cool to build. But more importantly, this is the best way I can contribute to what I think crypto’s ultimate goal should be: giving people anywhere in the world the opportunity to achieve financial freedom for their families.