This year, our team was tasked with creating a platform for publishers that would allow us to quickly and conveniently integrate with our advertising network. The platform should allow users to have a personal account with the ability to register, add and verify advertising placements and view statistics for various sections. I suggest we go through the path of creation step by step.

 

 

Choosing a technology stack

 

One of the most important steps to take when it comes to developing an application is choosing the right technology stack. This issue significantly affects the success of the project, both in the short and long term. Why? The technology stack for developing web applications affects project cost, time-to-market, security, and application scalability.

 

The service requirements at the design stage were:

• a stand-alone web service, Single Page Application.

• the ability to process a load of 100 requests per second (in peaks up to 200 rps).

• development and launch of Minimum Viable (MVA) product stage within 2 quarters.

• virtualization and containerization of the service with the ability to quickly deploy.

 

 

A well-chosen stack allows you to get a stable, safe, and easy-to-maintain product.

 

From an architectural point of view, web applications consist of two parts: client and server. The client-side is also called the frontend. This is essentially what users see on the device screen. The server part, or backend, is the software and hardware part of the service. It is a set of tools that implement the logic of an application.

 

Interaction between the client and server parts of a web application

 

There are three types of web applications that determine the approach to development:

• SPA or Single-Page Application is a single-page web application that loads into a single HTML page. Users don't need to reload or load additional pages thanks to dynamic refresh using JavaScript.

• MPA or Multi-Page Application are multi-page applications where every time the data changes or new information is loaded, the page is refreshed.

• A PWA or Progressive Web App is a kind of hybrid of a website with a mobile app.

 

Guided by the requirements and the need to speed up the service on the client's side as much as possible, we chose the first option - SPA. This allows us to update the content for the user dynamically as quickly as possible without reloading the page when interacting with the service.

 

Some of the most heavily loaded projects in the world use the same SPA approach: Netflix, Facebook, Gmail, e.t.c

 

 

Server-side of the application

 

Python and PHP are two of the most popular programming languages among web product developers. Both are powerful, both have a large following and active community.

 

Although they have much in common and are in demand among programmers, significant differences can also be found. Each of them serves different purposes: PHP was originally created for web development, while Python has a general purpose, it is more versatile. Which one is better is impossible to say for sure since everyone can find something useful for themselves in one of them.

 

For our team, the determining factor was that most of the services are already written in PHP and the developers have good experience with it. We had to decide only on the framework.

 

Of the most popular Laravel, Symfony and Yii, we settled on Laravel because it meets all the requirements (security, reliability, convenient REST API support, good documentation) and, lately, is almost synonymous with the expression "PHP framework". Its features: supports functional, integration, and unit testing, allow you to easily scale applications, follow development best practices, and offer a large selection of design patterns. This all helps to maintain a clean, minimalist, and efficient codebase that is easy to modify.

 

 

Client-side of the application

 

The choice of technology for the frontend came down to the choice of a JS framework for the project. The most popular ones for 2021 are ReactJS, AngularJS, and Vue. JS.

 

Although Vue is considered to be the best framework for Laravel, in past versions Laravel installed Vue support by default. But in the current versions, there is no such preset and we are invited to choose what to use for the frontend.

 

Since in other projects we use the UI Kit of Cốc Cốc Ad Platform, which provides React components and useful APIsUI Kit, we decided to use ReactJS.

 

Single Page Application which uses the latest version of Laravel 8.x
for RESTful back-end API. The frontend - components developed in React.

 

Database selection

 

To store user data (name, phone number, personal settings, etc.), it was necessary to select a reliable and fast database. Since the data format is predefined in advance, it is more convenient to use a relational database. The most popular and convenient at the present time, are MySQL and PostgreSQL. Their performance is about the same, and we did not plan to use Postgres' features (regular expressions, special data types).

 

The number of reads and writes to the database was expected to be small, the main load on statistics and reports data (more on this later).
We ended up with MysqlDB, using MariaDB Galera Cluster for fault tolerance.

 

For caching data, we traditionally used the Redis cluster. It is very fast and it is not critical for us to lose this data.

 

♦ Storage for statistics data

 

With statistics, things are much more interesting: there is a lot of data, and we have strict requirements for the time of generating reports - no more than 3 seconds per report for each publisher for a period of 3 months.

 

Since statistics data is usually not changed or deleted, we were faced with the task of choosing a very fast and reliable database for writing and reading data. The speed of changing and deleting data is not important.

 

Traditional relational databases weren't up to the task. After studying and comparing options, we settled on the ClickHouse open-source, high-performance columnar OLAP database.

 

Tests on preliminary data showed performance thousands of times better than Mysql and several times better than Vertica, a popular big data processing system:

 

Relative query processing time (lower is better)

 

Results for specific database queries

 

This performance allows us to generate statistics for 90 days for a specific user in less than 1 second.

 

 

Monitoring

 

No modern system can work stably without round-the-clock monitoring. Important metrics such as requests/errors per minute, the latency on GET/POST requests, the number of active PHP-FPM processes, and so on, should be monitored and alerts sent if the allowed values are exceeded.

 

For this, we use Grafana - the open-source analytics & monitoring solution platform.

 

 

Many graphs are displayed on large screens, which allows us to instantly respond to errors during deployment or a suddenly increased load on the service.

 

 

Conclusion

 

Thanks to the well-coordinated work of professional teams (product manager, project manager, frontend/backend developers, QA team), the project was launched into production in 2 quarters. Publishers register daily and actively add their domains for placing advertisements. We receive feedback from users and always try to take their opinion into account in the next updates of the platform.

 

Our team has big plans to improve and develop the advertising system, as well as many other interesting projects.

 

If you want to be a part of our team, and work on something meaningful and inspiring in a high-performance environment that empowers people - join us, we are waiting for you!