Build Your Own Yandex Maps Parser: Scrape Businesses, Reviews, and Phones Without Getting Blocked

Introduction: What You'll Get and Who This Guide Is For

Yandex Maps long ago evolved from a simple navigator into the most comprehensive business directory in Russia. Business listings contain exactly what every marketer, affiliate, or business owner needs: precise addresses, phone numbers, websites, opening hours, ratings, and thousands of honest customer reviews. The problem is that manually going through five hundred dental clinic listings in a single city is impossible, and off-the-shelf services are either expensive or deliver outdated data.

In this guide, we'll build our own Yandex Maps parser in Python together. No paid software, no third-party servers. By the end of this tutorial you'll have:

• A working script that scrapes business listings for any query and city.
• An Excel spreadsheet with names, categories, addresses, phones, websites, ratings, and opening hours.
• A separate table with reviews: author, rating, date, text.
• An understanding of how to work carefully without triggering captchas or blocks by distributing load through mobile proxies.

Important scope note for this article: we focus specifically on business listings inside Maps. Scraping regular Yandex search results, working with anti-detect browsers, and basic mobile proxy setup are covered in related blog posts — we won't repeat them here, just build on them.

Who This Guide Is For

• Marketers and business owners who need a database of competitors or partners in a region with real phone numbers and ratings.
• Affiliates gathering offline niches and local offers for further analysis.
• Developers tasked with building a Maps scraper who don't want to start from scratch.
• Analysts studying reviews to assess service quality in a niche.

What You Need to Know Beforehand

Nothing complicated. You just need to be able to install software, open a command line, and copy text. No programming experience required: all the code is ready, you just plug in your own values. If you've ever run a Python script, it'll be a breeze. In the advanced section we'll touch on async and working with network responses, but you can skip it.

How Much Time It Takes

• Environment setup: 30-40 minutes.
• Writing and debugging the parser step by step: 1.5-2 hours.
• First full scrape of 300-500 listings: 1-3 hours of background time while you do other things.

All in all, in half a workday you'll have a tool that will save you weeks of manual labor going forward.

Preliminary Setup: Tools, Access, and Requirements

Before writing code, let's gather everything we need. Skip this section only if you already have Python installed and access to a mobile proxy.

System Requirements

• Windows 10 or 11, macOS 12 or newer, or any modern Linux.
• At least 8 GB of RAM: we'll be running a real Chromium browser.
• 3-4 GB of free disk space for Python, the browser, and results.
• Stable internet. Speed isn't critical; no drops matter more.

What to Install

1. Python 3.11 or 3.12. Download the installer from the official python.org website. On Windows, be sure to check Add Python to PATH at the bottom of the first installer window, otherwise console commands won't work.
2. A code editor. Visual Studio Code, PyCharm Community, or even Notepad++ will do. We'll reference VS Code, but it makes no difference for our tasks.
3. Python libraries: playwright for browser control, pandas and openpyxl for spreadsheets, requests for checking the proxy. We'll install them in the first step.
4. Chromium browser for Playwright. Downloaded by a separate command, about 150 MB.

Access to a Mobile Proxy

This is the key element of the "no bans" section. Yandex Maps is sensitive to request frequency from a single address. Mobile proxies provide IP addresses of real mobile carriers, where thousands of regular users sit simultaneously, so Yandex treats them as favorably as possible. In your mobileproxy.space account after purchasing a proxy, you'll need four values:

• Host (server address) and port for HTTP connection.
• Username and password for authorization.
• IP change link: a special URL that, when requested, gives the proxy a new address from the carrier. Copy it to a separate file, you'll need it in step five.

Tip: Choose a proxy from the same region, or at least the same country, as the city you're scraping. Maps tailor results to geolocation, and a proxy from another country may show an incomplete list of businesses or an interface in a different language.

Backups and Working Folder

Create a folder on your disk, e.g. maps_parser. Scripts and results will live inside it. We'll save all intermediate data to files after each listing, so even if the script crashes on the three-hundredth business, the first two hundred ninety-nine won't be lost. It makes sense to back up the results file before each rerun: just rename the old file by adding the date.

Check: Open the command line (on Windows press Win+R, type cmd, and hit Enter) and run python --version. If you see something like Python 3.12.x, setup is done. If you get an error instead, reinstall Python with the Add to PATH option checked.

Basic Concepts: How Maps Works and What Listing Scraping Is

Before writing code, let's go over terminology. They're simple, but without them it won't be clear why we do things a certain way.

Key Terms in Plain Language

• Parsing (scraping) — automated collection of information from website pages. A program opens a page just like a human would and extracts the necessary fragments.
• Business listing — a separate page in Maps with an address like yandex.ru/maps/org/name/numeric-id/. That's where the phone, address, reviews, and other data live. The list of businesses on the left is just a storefront from which we take only links to listings.
• Selector — the "address" of an element within a page. For example, the class business-contacts-view__address points to the address block. The script finds the needed text by selectors.
• Headless browser — a real browser controlled by a program. Can work with a window (you see what's happening) or without one.
• Playwright — a library for controlling a browser from Python. We chose it because Maps is fully built on JavaScript, and a simple request for HTML would return an empty page without data.
• Mobile proxy — an intermediary between your computer and the website with a mobile network IP address. The site sees the carrier's address, not yours.
• IP rotation — periodic changing of the proxy address so that load doesn't concentrate on a single IP.
• Captcha — a "Confirm you're not a robot" verification page. For us it's a signal that we're going too fast. We won't solve it with third-party services, we'll just stop, change IP, and slow down.

Core Principles of a Yandex Maps Parser

The logic is simple and consists of three phases. First, we get a list of links to the listings we need. Then we open each listing one by one and extract the data. Finally, we put everything into a spreadsheet. Between these actions we take pauses of random duration and periodically change IP via the address change link.

What's Important to Understand About Legality and Ethics

Warning: Collect only publicly available business data and use it for analytics. A company's phone and address are not personal data, but review authors' names may qualify under Russian Federal Law 152-FZ. Don't store them unnecessarily and don't use collected phones for mass mailing without consent — that violates advertising law and Yandex's rules. Also remember: Yandex's user agreement restricts automated collection, so keep load minimal, and for commercial projects with large volumes, consider the official Yandex Business Search API.

Step 1: Setting Up the Environment and Connecting the Mobile Proxy

Goal of this stage: install all libraries, download the browser, and make sure requests go through the mobile proxy, not directly.

Installing Libraries

1. Open the command line or terminal.
2. Navigate to the working folder using cd and the folder path. For example: cd C:\maps_parser on Windows or cd ~/maps_parser on macOS and Linux.
3. Create a virtual environment so libraries don't interfere with other projects: python -m venv venv
4. Activate it. Windows: venv\Scripts\activate. macOS and Linux: source venv/bin/activate. You'll see (venv) at the start of the terminal line.
5. Install the libraries with a single command:

pip install playwright pandas openpyxl requests
playwright install chromium

The second command downloads Chromium. This takes 2-5 minutes depending on internet speed. Wait until the terminal shows the input prompt again.

Checking the Proxy

In your editor, create a file called check_proxy.py and paste the code below, replacing LOGIN, PASSWORD, HOST, and PORT with the values from your account:

import requests

proxy = 'http://ЛОГИН:ПАРОЛЬ@ХОСТ:ПОРТ'
proxies = {'http': proxy, 'https': proxy}

direct = requests.get('https://api.ipify.org?format=json', timeout=30).json()
via_proxy = requests.get('https://api.ipify.org?format=json', proxies=proxies, timeout=30).json()
print('Ваш прямой IP:', direct['ip'])
print('IP через прокси:', via_proxy['ip'])

Run the file with python check_proxy.py. You should see two different addresses. If they match or a connection error appears, the proxy isn't working and there's no point going further.

Checking IP Change

Open the IP change link from your account in a regular browser. Usually it returns a short success message. Wait 15-30 seconds and run check_proxy.py again. The address via the proxy should have changed. Note how many seconds the change actually takes: we'll plug that value into the script in step five.

Tip: Save all proxy settings to a separate config.py file with variables PROXY_HOST, PROXY_PORT, PROXY_LOGIN, PROXY_PASSWORD, and CHANGE_IP_URL. Then in other scripts you just write from config import * and don't retype passwords ten times.

Check: The command playwright --version outputs the version number, check_proxy.py shows a mobile carrier IP that differs from your home one, and after hitting the change link the address changes.

Possible Issues

• Error "pip is not recognized as an internal command." Python was installed without adding to PATH. Reinstall with the checkbox or use py -m pip instead of pip.
• Error 407 Proxy Authentication Required. Wrong login or password. Check for stray spaces when copying.
• Timeout when requesting through the proxy. Wrong port, or the proxy is currently changing IP. Wait half a minute and retry.

Step 2: Collecting Links to Business Listings

Goal of this stage: get a links.json file with a list of listing URLs for all businesses matching the query in the target city.

Here we'll peek into the Maps results list only once, to grab the links. We'll take the data itself exclusively from the listings, so the list is just a table of contents, nothing more.

How the Business List Is Structured

When you enter a query like "dentist Kazan" into Maps, a scrollable panel of snippets appears on the left. Each snippet contains a link to a listing. The list loads in chunks as you scroll, so the script will spin the panel with the mouse wheel and collect links after each scroll until new ones stop appearing.

Writing the Link Collection Script

Create a file called collect_links.py:

from playwright.sync_api import sync_playwright
import time, random, json
from config import *

PROXY = {'server': f'http://{PROXY_HOST}:{PROXY_PORT}', 'username': PROXY_LOGIN, 'password': PROXY_PASSWORD}
QUERY = 'стоматология Казань'
MAX_SCROLLS = 40

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False, proxy=PROXY)
    context = browser.new_context(locale='ru-RU', viewport={'width': 1400, 'height': 900})
    page = context.new_page()
    page.goto('https://yandex.ru/maps/?text=' + QUERY, wait_until='domcontentloaded')
    time.sleep(random.uniform(5, 8))
    page.mouse.move(350, 500)
    links = set()
    stale = 0
    for i in range(MAX_SCROLLS):
        before = len(links)
        page.mouse.wheel(0, random.randint(1200, 2000))
        time.sleep(random.uniform(1.5, 3.5))
        for a in page.query_selector_all('a[href*="/maps/org/"]'):
            href = a.get_attribute('href') or ''
            clean = href.split('?')[0]
            if clean.startswith('/'):
                clean = 'https://yandex.ru' + clean
            links.add(clean)
        stale = stale + 1 if len(links) == before else 0
        print(f'Прокрутка {i+1}: ссылок {len(links)}')
        if stale >= 4:
            break
    with open('links.json', 'w', encoding='utf-8') as f:
        json.dump(sorted(links), f, ensure_ascii=False, indent=2)
    print('Итого собрано:', len(links))
    browser.close()

Breaking Down What Happens

1. The line headless=False opens the browser with a window. During debugging this is essential: you'll see for yourself whether the map loaded and whether there's a captcha.
2. page.mouse.move(350, 500) puts the cursor on the left panel. Without this, the mouse wheel would scroll the map, not the list.
3. The selector a[href*="/maps/org/"] finds all links containing the /maps/org/ fragment. It's more resilient than specific classes that Yandex changes every few months.
4. The stale counter stops the loop if four consecutive scrolls brought no new links. That means the list is exhausted.
5. Links are stripped of parameters after the question mark so the same business doesn't end up in the file twice.

Run the script: python collect_links.py. A browser window opens, the map with results loads, the panel starts scrolling. The link counter ticks up in the terminal. For an average city, one query usually yields 100 to 400 listings in 2-4 minutes.

Tip: Maps rarely returns more than 500 results per query. If you need the whole niche in a megacity, split the query by district: "dentist Vakhitovsky district Kazan," "dentist Sovetsky district Kazan." Combine links from different queries via set, as in the code above — duplicates disappear by themselves.

Check: The links.json file appears in the folder, containing a list of addresses like https://yandex.ru/maps/org/name/1234567890/. Open two or three addresses manually in a regular browser and make sure they're the right business listings.

Possible Issues

• Zero links collected. Most likely the page didn't finish loading or the cursor wasn't over the list. Increase the first pause to 10 seconds and check the mouse coordinates: the panel must be under the cursor.
• The list doesn't scroll, the map moves. Adjust the X coordinate in mouse.move, based on the panel width in your window.
• Captcha appears immediately. Change IP via the link, wait a minute, and run again. If it repeats, try a different proxy channel.

Step 3: Scraping a Business Listing — Name, Address, Phone, Website

Goal of this stage: write a function that opens a single listing and returns a dictionary with the business's basic data, and run it over all links from links.json.

How to Find Selectors Yourself

Yandex periodically renames classes in its markup. So it's important to be able to find them yourself, not only rely on ready-made code. Here's how:

1. Open any business listing in a regular Chrome browser.
2. Right-click on the business's phone number and choose Inspect (or press F12 and click an element with the pick tool).
3. In the panel that opens, you'll see the highlighted tag with a class attribute. For example, class="card-phones-view__phone-number". That's the selector: in code it's written with a dot at the start.
4. Repeat for the name, address, website, rating, and opening hours. Write the classes in a notepad.

A class may consist of several words separated by spaces. Take the first, most "tell-tale" one, with a double underscore inside. At the time of writing, the selectors in the code below are current, but check them before running.

Writing the Listing Parsing Function

Create a file called parse_cards.py:

from playwright.sync_api import sync_playwright
import time, random, json, csv, os
from config import *

PROXY = {'server': f'http://{PROXY_HOST}:{PROXY_PORT}', 'username': PROXY_LOGIN, 'password': PROXY_PASSWORD}
OUT = 'orgs.csv'
FIELDS = ['url', 'name', 'category', 'address', 'phones', 'site', 'rating', 'reviews_count', 'hours']

def grab(page, selector):
    el = page.query_selector(selector)
    return el.inner_text().strip() if el else ''

def parse_card(page, url):
    page.goto(url, wait_until='domcontentloaded')
    time.sleep(random.uniform(3, 6))
    more = page.query_selector('.card-phones-view__more')
    if more:
        more.click()
        time.sleep(random.uniform(1, 2))
    phones = [e.inner_text().strip() for e in page.query_selector_all('.card-phones-view__phone-number')]
    return {
        'url': url,
        'name': grab(page, 'h1.orgpage-header-view__header'),
        'category': grab(page, '.orgpage-categories-info-view'),
        'address': grab(page, '.business-contacts-view__address'),
        'phones': '; '.join(dict.fromkeys(phones)),
        'site': grab(page, '.business-urls-view__text'),
        'rating': grab(page, '.business-rating-badge-view__rating-text'),
        'reviews_count': grab(page, '.business-header-rating-view__text'),
        'hours': grab(page, '.business-working-status-view'),
    }

def load_done():
    if not os.path.exists(OUT):
        return set()
    with open(OUT, encoding='utf-8') as f:
        return {row['url'] for row in csv.DictReader(f)}

links = json.load(open('links.json', encoding='utf-8'))
done = load_done()
todo = [u for u in links if u not in done]
print(f'Всего {len(links)}, осталось {len(todo)}')

with sync_playwright() as p:
    browser = p.chromium.launch(headless=False, proxy=PROXY)
    page = browser.new_context(locale='ru-RU').new_page()
    new_file = not os.path.exists(OUT)
    with open(OUT, 'a', newline='', encoding='utf-8') as f:
        writer = csv.DictWriter(f, fieldnames=FIELDS)
        if new_file:
            writer.writeheader()
        for n, url in enumerate(todo, 1):
            try:
                row = parse_card(page, url)
                writer.writerow(row)
                f.flush()
                print(f'{n}/{len(todo)} {row["name"]} | {row["phones"]}')
            except Exception as e:
                print('Ошибка на', url, e)
            time.sleep(random.uniform(4, 9))
    browser.close()

What's Important Here

1. The "Show phone" button. In some listings the full number is hidden behind a button. We look for it by the class card-phones-view__more and click if found. Only then do we collect numbers.
2. Writing after each listing. The f.flush() call forces data to disk. If the script crashes, everything collected stays in orgs.csv.
3. Resuming work. The load_done function reads already-collected links, and on rerun the script continues where it left off instead of starting over.
4. Pauses of 4-9 seconds between listings. This is the pace of an attentive human reading information. Don't reduce it on first runs.

Run python parse_cards.py and watch the first five or ten listings in the browser window. You should see the page open, the phone expand if needed, and the name and number appear in the terminal.

Warning: If five consecutive listings return empty names, immediately stop the script with Ctrl+C. Almost certainly Yandex changed the markup and the selectors are outdated. Find new ones per the instructions above and update the code. Continuing to collect empty rows is pointless and only increases proxy load.

Tip: Besides text, it's useful to save the business's ID: it's the last numeric part of the URL. It makes it easy to compare databases between scrapes and track closed companies. Add the field 'org_id': url.rstrip('/').split('/')[-1] to the dictionary.

Check: The orgs.csv file opens in Excel, with name, address, phones filled in for at least 90 percent of rows. Phones display in the format +7 (843) 000-00-00. The rating looks like a number with a comma, e.g. 4,7.

Possible Issues

• Phones are empty although they exist on the site. The "Show phone" button has a different class. Find it via F12 and replace it in the code.
• Cyrillic in CSV shows up as gibberish. Excel didn't recognize the encoding. Open the file via Data > From Text/CSV and choose UTF-8, or wait for step 6, where we convert data to xlsx.
• The script hangs on one listing. Add the parameter timeout=45000 to page.goto so an exception is raised after 45 seconds and the loop moves on.

Step 4: Collecting Reviews from Listings

Goal of this stage: for each business, get a list of reviews with rating, date, and text, and save them to a separate reviews.csv file.

Where Reviews Live

Each listing has a "Reviews" tab with an address like https://yandex.ru/maps/org/name/id/reviews/. Reviews load in chunks as you scroll, just like the business list in step two. By default they're sorted by relevance, and you can get fresh ones by switching sorting to "Newest."

Writing the Review Collection Function

Create a file called parse_reviews.py. The basis is the same as in the previous step, so we'll only show the function and loop:

def parse_reviews(page, url, max_scrolls=15):
    page.goto(url.rstrip('/') + '/reviews/', wait_until='domcontentloaded')
    time.sleep(random.uniform(3, 6))
    page.mouse.move(350, 600)
    seen = 0
    for _ in range(max_scrolls):
        page.mouse.wheel(0, random.randint(1500, 2500))
        time.sleep(random.uniform(1.5, 3))
        cards = page.query_selector_all('.business-review-view')
        if len(cards) == seen:
            break
        seen = len(cards)
    result = []
    for c in page.query_selector_all('.business-review-view'):
        def sub(sel):
            el = c.query_selector(sel)
            return el.inner_text().strip() if el else ''
        stars = c.query_selector_all('.business-rating-badge-view__star._full')
        result.append({
            'org_url': url,
            'author': sub('.business-review-view__author-name'),
            'date': sub('.business-review-view__date'),
            'stars': len(stars),
            'text': sub('.business-review-view__body-text'),
        })
    return result

In the loop over links, call parse_reviews instead of parse_card and write each list item as a separate row in reviews.csv with fields org_url, author, date, stars, text. The resume logic and writing after each business stay the same.

Breaking Down Details

1. Limit max_scrolls=15. Popular places can have two or three thousand reviews. Collecting them all is rarely needed, and it eats up lots of time and requests. Fifteen scrolls usually gets 100-150 recent reviews.
2. Rating via stars. The score isn't written as text in the review but drawn as stars. We count elements with the _full modifier, i.e. filled-in stars.
3. Long reviews. Some texts are collapsed and require clicking "More." If you need the full text, before collecting click all buttons with the class business-review-view__expand inside the card.

Warning: Review authors' names are user data, not business data. If your task is sentiment analysis or finding typical complaints, you don't need the author field. Don't collect it without purpose — that spares you unnecessary questions under 152-FZ. If you do need it, keep the file local and don't share it with third parties.

Tip: Run review collection not over all businesses but over a filtered list: e.g. only those with a rating below 4.0 or only direct competitors. That cuts request volume manifold without losing the value of the data.

Check: In reviews.csv each business has 20 to 150 rows, the stars column contains numbers from 1 to 5, and text has meaningful Russian text. The date looks like "15 января" or "3 марта 2026."

Possible Issues

• Zero reviews found. Check that the URL ends in /reviews/ and the reviews panel is under the cursor when scrolling.
• All reviews have stars equal to 0. The filled star class changed. Find it via F12 by clicking on a star.
• Reviews are duplicated. The panel didn't scroll fully and one block was counted twice. Remove duplicates in step six by the pair author plus text.

Step 5: Setting Up IP Rotation and Block Protection

Goal of this stage: teach the parser to behave like a careful user: change IP on schedule, recognize captchas, and automatically slow down instead of hitting a block.

Why Bans Happen at All

Yandex doesn't forbid viewing listings, but it watches for anomalies: hundreds of pages a minute from one address, identical intervals between requests, no mouse movement, empty cookies. When suspicion accumulates, the SmartCaptcha page appears, and with persistence, a temporary IP restriction. Our strategy isn't to "break through" the defense but to give no reason for it to trigger.

Three Rules of a Calm Yandex Maps Parser

1. Human pace. No more than 8-12 listings per minute per IP. Random pauses, not fixed ones.
2. Regular IP change. Every 25-40 listings or every 10-15 minutes, hit the address change link. The mobile proxy gets a new carrier IP in seconds, and the request history is "reset" from the site's point of view.
3. Immediate rollback on captcha. See a verification — don't try to solve it, pause for 2-3 minutes, change IP, and continue from the same listing in a new browser context.

Adding Rotation to the Code

Paste the following functions into parse_cards.py and parse_reviews.py and call them in the main loop:

import requests

def change_ip():
    try:
        r = requests.get(CHANGE_IP_URL, timeout=30)
        print('Смена IP:', r.status_code)
    except Exception as e:
        print('Не удалось сменить IP:', e)
    time.sleep(IP_CHANGE_WAIT)

def is_captcha(page):
    if 'showcaptcha' in page.url or 'checkcaptcha' in page.url:
        return True
    return page.query_selector('.CheckboxCaptcha') is not None

def new_page(browser):
    ctx = browser.new_context(locale='ru-RU', viewport={'width': random.choice([1366, 1440, 1536]), 'height': 900})
    return ctx.new_page()

And the main loop takes this form:

page = new_page(browser)
since_change = 0
for n, url in enumerate(todo, 1):
    if since_change >= random.randint(25, 40):
        page.context.close()
        change_ip()
        page = new_page(browser)
        since_change = 0
    row = parse_card(page, url)
    if is_captcha(page):
        print('Капча! Пауза и смена IP')
        page.context.close()
        time.sleep(random.uniform(120, 180))
        change_ip()
        page = new_page(browser)
        row = parse_card(page, url)
    writer.writerow(row)
    f.flush()
    since_change += 1
    time.sleep(random.uniform(4, 9))

What's Important to Understand

• New context along with new IP. Closing the context resets cookies and local storage. Changing the address without resetting cookies is pointless: the site will keep recognizing you by session.
• The IP_CHANGE_WAIT variable in config.py is the time measured in step one, usually 15-30 seconds. You can't set less: the browser will open the page through the old address.
• Random window size adds variety to the browser fingerprint. It's a soft measure, but it's free.
• Mobile proxies with link-based rotation are more convenient here than anything else: you don't switch between dozens of addresses manually, you just hit one URL.

Tip: Keep a simple log: record the time, listing number, and event (success, captcha, IP change) to a file. After a week, the log will tell you exactly at what pace captcha never appears, and you can tune pauses to your proxy channel.

Check: After an hour of continuous work the script collected 400-600 listings, with no more than one or two captcha messages in the terminal, and collection resumed automatically after each. The IP via the proxy changed at least ten times (visible in the "IP change: 200" lines).

Possible Issues

• Captcha appears every 10 listings. The pace is too high for your channel. Increase pauses to 8-15 seconds and change IP every 15 listings.
• Pages don't load after IP change. Increase IP_CHANGE_WAIT: the carrier hasn't issued a new address yet.
• The IP change link returns an error about frequent requests. Most plans have a minimum interval between changes, usually one to two minutes. Don't change IP more often.

Step 6: Cleaning Data and Saving to Excel

Goal of this stage: turn raw CSVs into tidy Excel tables without duplicates, with normalized phone numbers and numeric ratings.

Writing the Cleaning Script

Create a file called clean_export.py:

import pandas as pd

def norm_phone(value):
    out = []
    for raw in str(value).split(';'):
        digits = ''.join(ch for ch in raw if ch.isdigit())
        if len(digits) == 11 and digits[0] in '78':
            out.append('+7' + digits[1:])
        elif len(digits) == 10:
            out.append('+7' + digits)
    return '; '.join(dict.fromkeys(out))

orgs = pd.read_csv('orgs.csv', encoding='utf-8')
orgs = orgs.drop_duplicates(subset='url')
orgs['phones'] = orgs['phones'].fillna('').apply(norm_phone)
orgs['rating'] = pd.to_numeric(orgs['rating'].astype(str).str.replace(',', '.'), errors='coerce')
orgs['reviews_count'] = pd.to_numeric(orgs['reviews_count'].astype(str).str.extract('(\d+)')[0], errors='coerce')
orgs = orgs.sort_values('rating', ascending=False)

reviews = pd.read_csv('reviews.csv', encoding='utf-8')
reviews = reviews.drop_duplicates(subset=['org_url', 'author', 'text'])

with pd.ExcelWriter('yandex_maps_result.xlsx', engine='openpyxl') as w:
    orgs.to_excel(w, sheet_name='Организации', index=False)
    reviews.to_excel(w, sheet_name='Отзывы', index=False)
print('Организаций:', len(orgs), 'Отзывов:', len(reviews))

In the line that extracts the review count, a regular expression is used with a single backslash and the letter d in parentheses: it pulls out the first number from text like "312 отзывов." Copy it carefully.

What the Script Does

1. Removes duplicate businesses by URL.
2. Normalizes all phones to the format +7XXXXXXXXXX, removing brackets, spaces, and hyphens. This format is convenient for CRM and database matching.
3. Replaces the comma with a dot in the rating so Excel treats it as a number and allows sorting.
4. Extracts the review count from the text string.
5. Removes duplicate reviews and writes two sheets into a single xlsx file.

Run python clean_export.py and open yandex_maps_result.xlsx. On the first sheet, businesses are sorted by rating, phones are uniform, and on the second sheet — reviews linked to businesses by the org_url column.

Tip: Add a "scrape date" column with the current date to the script. In a month, repeat the scrape and compare tables with pandas' merge function: you'll see new businesses, closed locations, and changes in competitors' ratings. That's already full-fledged market monitoring.

Check: The xlsx file opens without warnings, Cyrillic is readable, the rating column sorts as a number, the phones column has no brackets or spaces, and the number of rows on the "Businesses" sheet matches the number of unique links in links.json minus errors.

Result Check: A Ready Parser Checklist

Go through the list. If you answered "yes" to each point, your Yandex Maps parser is ready for regular work.

Checklist

• check_proxy.py shows a mobile carrier IP different from your home one.
• collect_links.py collects more than 50 links on an average query and stops by itself.
• parse_cards.py fills in name, address, and phone for at least 90 percent of listings.
• The "Show phone" button expands automatically.
• parse_reviews.py returns reviews with a rating from 1 to 5.
• On captcha, the script pauses, changes IP, and continues without your involvement.
• After an emergency stop, a rerun continues from the break point.
• clean_export.py creates an xlsx with two sheets and normalized phones.

How to Test End-to-End

1. Take a small query with 30-60 businesses in the city, e.g. "tire service" in a district center.
2. Run all scripts in sequence and time it. 50 listings with reviews should take 15-25 minutes.
3. Pick five random businesses from the table and verify phones and addresses against the listings manually.
4. Stop parse_cards.py midway with Ctrl+C and run it again. Make sure the "remaining" counter decreased, not reset.

Indicators of Success

• Data accuracy on manual verification: 100 percent matches for phones and addresses.
• Share of empty names: less than 3 percent.
• Captcha frequency: no more than one per 200-300 listings.
• Speed: 400-600 listings per hour per proxy channel at a safe pace.

Common Mistakes and How to Fix Them

We've gathered the problems nearly everyone runs into when first writing a Yandex Maps parser and ways to fix them.

Empty Fields in Most Listings

Cause: Yandex updated the markup, element classes changed. Fix: open a listing in Chrome, press F12, find the new classes, and replace them in parse_card. Keep selectors in one dictionary at the top of the file so you edit in one place.

Captcha Appears on the Very First Page

Cause: The proxy IP is already "tired" from previous activity, or the browser launches with suspicious parameters. Fix: change IP before starting, make sure locale='ru-RU' is set, and don't use headless mode on first runs: it gives more automation signals.

Scrolling Moves the Map, Not the Panel

Cause: The mouse cursor is outside the left panel. Fix: adjust the page.mouse.move coordinates for your window size. At 1400 pixels wide, the panel takes roughly the first 450 pixels horizontally.

The Same 20 Businesses Keep Getting Collected

Cause: The panel doesn't scroll to the end, and the stale counter triggers too early. Fix: increase the pause after scrolling to 3-4 seconds and the stale threshold to 6 — Maps sometimes loads the next chunk slowly.

Phones Are Visible in the Browser but Empty in the Table

Cause: The number appears only after clicking, and the script collects data before it finishes, or the button has a different class. Fix: increase the pause after clicking to 2-3 seconds and check the button class.

Target Closed or Browser Has Been Closed Error

Cause: You closed the context during IP change but keep using the old page object. Fix: make sure that after each page.context.close() the page variable is reassigned via new_page(browser), as in the step 5 example.

Pages Load Forever After IP Change

Cause: The carrier hasn't allocated a new address yet, proxy is in a transitional state. Fix: increase IP_CHANGE_WAIT to 30-40 seconds and add a timeout to page.goto so a hang doesn't block the loop.

Excel Opens CSV with Gibberish

Cause: Excel can't recognize UTF-8 without a BOM. Fix: use clean_export.py and work with xlsx, or specify encoding='utf-8-sig' when writing the CSV.

Advanced Features for Pros

The basic parser already solves 90 percent of tasks. If you want more speed and data, here's where to grow.

Intercepting Network Responses Instead of Parsing Markup

Maps loads listing data as JSON in separate requests. Playwright can subscribe to them via page.on('response', handler). Inside the handler, check whether response.url contains the /maps/api/ fragment and save response.json(). Pros: data is structured and doesn't depend on markup classes. Cons: internal addresses change without warning, and parsing nested JSON is harder than reading text off the page. The approach pairs well with the main one: if the JSON response arrives — use it, otherwise fall back to HTML parsing.

Parallel Work with Multiple Proxy Channels

One mobile proxy at a safe pace yields 400-600 listings per hour. If you need faster, buy two or three channels and run a separate process for each, splitting links.json into equal parts. Don't run multiple browsers through one channel: the total rate per IP will rise and captcha will return. For orchestration, a simple launcher script on subprocess or the asyncio library with async_playwright will do, where each worker gets its own context and its own proxy in the proxy parameter of new_context.

Change Monitoring

Save the results of each scrape to a separate file with a date and compare them by org_id. New IDs are new market players, missing ones are closed businesses, changes in rating and reviews_count are reputation dynamics. Schedule a run every two weeks via Windows Task Scheduler or cron, and you'll have a living map of the niche.

Extending Fields

Listings also have other useful blocks: a services list with prices, social media links, a "Verified business" flag, photo count, nearest metro. Each field is added the same way: find the class via F12, add a grab to the dictionary. The price block is especially valuable for affiliates assessing the average check in a niche.

Review Analysis

Collected texts work great for simple analytics: count word frequency in reviews with one or two stars and get a list of the top customer complaints about competitors. pandas and Counter from the standard library are enough. An advanced option is to run texts through a language model for topic classification: price, quality, service, wait.

Official API as an Alternative

For large commercial projects, consider Yandex's Business Search API. It returns name, address, phone, and categories in a structured form and carries no block risks at all. It has no reviews, limits are paid, but for building a contact database, it's the cleanest path. The listing parser then remains a tool for reviews and fields the API doesn't have.

FAQ: Common Questions About Yandex Maps Parsing

Is it legal to scrape business phones from Yandex Maps?

Company contact data is published publicly by the businesses themselves and isn't personal data. Collecting it for your own analysis is acceptable. But using these numbers for mass calls and mailings without consent violates advertising law. Also remember Yandex's user agreement restrictions on automated collection and keep load minimal.

Why can't we just use regular requests without a browser?

Maps is fully rendered by JavaScript. The server returns nearly empty HTML, and data loads later. requests would get the frame without phones and addresses. That's why we need Playwright with real Chromium.

Is a mobile proxy required, or can I scrape from my home IP?

Technically the first 50-100 listings will be collected just fine. But then captcha will appear, and your home IP will be restricted for several hours, preventing you from using Yandex normally. Mobile proxies solve the problem in two ways: a mobile carrier address is inherently more trusted, and link-based rotation lets you distribute load across addresses without stopping the scrape.

How many listings can I collect per day from one proxy?

At a safe pace of 8-12 listings per minute with pauses and IP change every 30 listings — about 5-8 thousand per day of continuous work. With reviews included, the volume drops two- to threefold because each reviews page requires scrolling.

What if Yandex changes its markup and the parser breaks?

That's a routine situation, happens a few times a year. Open a listing, press F12, find the new classes for the needed elements, and replace them in the code. Takes 10-15 minutes. To make it easier, keep all selectors in one dictionary at the top of the file.

How do I collect businesses across an entire region rather than one city?

Compile a list of localities and run collect_links.py in a loop, substituting the name into QUERY. Combine links into a single set. For large cities, additionally split by district, since one query rarely returns more than 500 results.

Can I run the parser in the background without a browser window?

Yes, set headless=True. But do this only after you've debugged selectors and confirmed captcha doesn't appear. In windowless mode, problems are harder to notice, and some automation indicators show up more strongly. Compromise: headless=True plus a page screenshot on every error via page.screenshot(path='error.png').

How do I know the script hit a captcha if I'm not watching the screen?

The is_captcha function from step 5 checks the page URL and the presence of the verification block. Add a notification to yourself, e.g. a log file entry or a messenger message via a bot, and you'll learn about the problem immediately.

Reviews are only partially collected, just the latest hundred. How do I get more?

Increase max_scrolls in parse_reviews to 50-100. Note that each scroll is an additional request to the server, so for businesses with thousands of reviews, collection will take several minutes and require more frequent IP changes.

How is this better than off-the-shelf scraping services?

You fully control the fields, pace, and freshness of data, don't pay per row, and don't depend on someone else's update schedule. Off-the-shelf services are convenient for a one-off task of a few hundred listings, while your own Yandex Maps parser pays for itself by the second scrape.

Conclusion

Let's wrap up. You installed Python and Playwright, connected a mobile proxy, and verified IP change. Collected links to business listings for a query and city. Wrote a function that opens each listing, expands the hidden phone, and grabs the name, address, website, rating, and opening hours. Added review collection with ratings. Taught the parser to change IP on schedule and respond correctly to captcha. Finally, cleaned the data and got a tidy Excel with two sheets.

The main thing to remember: the resilience of a Yandex Maps parser rests not on tricks but on three things — human pace, regular address rotation via mobile proxies, and a willingness to stop at the first signal. A script that respects the resource runs for months without intervention.

What to Do Next

• Run your first full scrape in your niche and verify five to ten listings manually.
• Schedule a scrape every two weeks and start accumulating a history of changes.
• Try intercepting JSON responses from the advanced section to depend less on markup.
• If volumes grow, add a second proxy channel and parallelize work.

Where to Grow

The next logical step is automatic analysis of collected reviews and linking the table to your CRM or ad account. And if you work with multiple platforms, the same approach with Playwright and mobile proxies transfers to any site with dynamic loading. The principles are the same, only the selectors change. Happy scraping and clean data!