LEARN COMPLETE PYTHON IN 24 HOURS

9. Concurrency & Parallelism

9.1 Threading vs Multiprocessing vs Asyncio – Quick Comparison

ApproachUses threads?Uses processes?Best forGIL effectOverheadTypical use casethreadingYesNoI/O-bound (waiting: network, files, DB)Limited (GIL blocks CPU)LowSimple background tasks, many file readsmultiprocessingNoYesCPU-bound (heavy math, image processing)No GIL per processHigh (memory)Parallel computation, ML trainingasyncioNo (co-op)NoHigh-volume I/O (thousands of requests)No issueVery lowWeb servers, API clients, scraping

GIL (Global Interpreter Lock) short explanation: CPython has one GIL → only one thread can execute Python bytecode at a time. → Threads are great for waiting (I/O), but almost no speedup for CPU work. → Multiprocessing creates separate processes → each has its own GIL → true multi-core usage.

2026 decision guide:

• Need to wait for 100+ network calls? → asyncio (best) or threading

• Need to run heavy math on all CPU cores? → multiprocessing

• Simple script with few background tasks? → threading

• Modern web/API/scraping? → asyncio

9.2 threading module – Basics & Locks

threading is good when your bottleneck is waiting (network, disk, database), not CPU.

Basic example – concurrent downloads (simulated)

Python

import threading import time def download_file(file_id, delay): print(f"Starting download {file_id}") time.sleep(delay) # simulate network delay print(f"Finished download {file_id}") threads = [] for i in range(1, 6): t = threading.Thread(target=download_file, args=(i, 2.0)) threads.append(t) t.start() for t in threads: t.join() # wait for all threads to finish print("All downloads completed")

Race condition problem (shared variable)

Python

counter = 0 def increment(): global counter for in range(100000): temp = counter time.sleep(0.000001) # simulate tiny delay counter = temp + 1 threads = [threading.Thread(target=increment) for in range(10)] for t in threads: t.start() for t in threads: t.join() print(counter) # Expected 1000_000, but usually much less!

Solution: Lock (Mutex)

Python

counter = 0 lock = threading.Lock() def safe_increment(): global counter for in range(100000): with lock: # automatically acquire & release counter += 1 # Now counter == 1_000_000 every time

Other tools in threading:

• threading.RLock → reentrant lock (same thread can acquire multiple times)

• threading.Event, threading.Condition, threading.Semaphore

• queue.Queue → thread-safe queue (very useful)

Modern recommendation: Use concurrent.futures.ThreadPoolExecutor for simple thread pools.

9.3 multiprocessing – Process Pools

multiprocessing runs real OS processes → bypasses GIL → true parallelism on multiple CPU cores.

Important: Always protect main code with if name == "__main__": (especially on Windows)

Basic Process example

Python

from multiprocessing import Process def square(n): print(f"{n}² = {n*n}") if name == "__main__": processes = [Process(target=square, args=(i,)) for i in range(10)] for p in processes: p.start() for p in processes: p.join()

Best & easiest way: ProcessPoolExecutor

Python

from concurrent.futures import ProcessPoolExecutor import time def heavy_math(n): time.sleep(1) # simulate CPU work return n * n if name == "__main__": with ProcessPoolExecutor(max_workers=4) as executor: results = executor.map(heavy_math, range(10)) print(list(results)) # [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]

Shared data (be careful – slow & complex): Use multiprocessing.Value, Array, Manager, Queue, Pipe

Tip: For CPU-bound tasks → multiprocessing gives real speedup on multi-core machines.

9.4 asyncio – Async/Await Syntax

asyncio uses cooperative multitasking in a single thread — extremely efficient for I/O-heavy workloads.

Basic async example

Python

import asyncio async def fetch_data(source_id): print(f"Fetching from {source_id}...") await asyncio.sleep(1.5) # simulate network delay print(f"Done with {source_id}") return f"Data from {source_id}" async def main(): tasks = [fetch_data(i) for i in range(1, 6)] results = await asyncio.gather(*tasks) # run all concurrently print("All results:", results) if name == "__main__": asyncio.run(main()) # Python 3.7+ standard way

Output (finishes in ~1.5 seconds, not 7.5):

text

Fetching from 1... Fetching from 2... Fetching from 3... Fetching from 4... Fetching from 5... Done with 1 Done with 2 Done with 3 Done with 4 Done with 5 All results: ['Data from 1', 'Data from 2', 'Data from 3', 'Data from 4', 'Data from 5']

Key concepts:

• async def → defines a coroutine

• await → pauses coroutine until awaited task completes

• asyncio.gather() → wait for multiple tasks concurrently

• asyncio.create_task() → schedule without immediate await

9.5 aiohttp, async file I/O

aiohttp – asynchronous HTTP client/server (replaces requests for async code)

Install: pip install aiohttp

Concurrent HTTP requests

Python

import asyncio import aiohttp async def fetch(url): async with aiohttp.ClientSession() as session: async with session.get(url) as response: return await response.text() async def main(): urls = [ "https://python.org", "https://fastapi.tiangolo.com", "https://realpython.com" ] tasks = [fetch(url) for url in urls] results = await asyncio.gather(*tasks) print([len(html) for html in results]) # lengths of pages asyncio.run(main())

Async file reading/writing (aiofiles)

Python

# pip install aiofiles import aiofiles async def read_large_file(): async with aiofiles.open("big_log.txt", mode="r", encoding="utf-8") as f: async for line in f: # process line if "ERROR" in line: print(line.strip()) asyncio.run(read_large_file())

9.6 When to Use What (GIL Explanation)

GIL reminder: CPython has one Global Interpreter Lock → only one thread executes Python bytecode at a time. → I/O operations (network, disk) release GIL → threads work well → CPU-bound Python code keeps GIL → threads give almost no speedup

Practical 2026 decision table:

ScenarioRecommendedWhy / Alternative100+ API calls / web scrapingasyncio + aiohttpThousands concurrent, low CPU usageReading/writing many files or DB queriesasyncio + aiofilesNon-blocking I/OHeavy math, image processing, ML inferencemultiprocessingUses all CPU cores, bypasses GILSimple background tasks (email, logging)threadingEasy, low overheadWeb server (high connections)asyncio (FastAPI, aiohttp)Best scalabilityMixed I/O + CPU workasyncio + ProcessPoolExecutorAsync I/O + CPU offloaded to processes

Mini Project – Concurrent API Caller with asyncio

Python

import asyncio import aiohttp from aiohttp import ClientTimeout async def get_user(user_id): url = f"https://jsonplaceholder.typicode.com/users/{user_id}" timeout = ClientTimeout(total=5) try: async with aiohttp.ClientSession() as session: async with session.get(url, timeout=timeout) as resp: return await resp.json() except Exception as e: return {"error": str(e)} async def main(): tasks = [get_user(i) for i in range(1, 11)] results = await asyncio.gather(*tasks, return_exceptions=True) for user in results: if isinstance(user, dict) and "name" in user: print(f"{user['id']}: {user['name']}") else: print("Failed:", user) asyncio.run(main())

This completes the full Concurrency & Parallelism section — now you know how to choose and implement the right concurrency model for any task!

📚 Amazon Book Library

All my books are FREE on Amazon Kindle Unlimited🌍 Exclusive Country-Wise Amazon Book Library – Only Here!

On GlobalCodeMaster.com you’ll find complete, ready-to-use lists of my books with direct Amazon links for every country.

Belong to India, Australia, USA, UK, Canada or any other country? Just click your country’s link and enjoy:

✅ Any eBook FREE on Kindle Unlimited ✅ Or buy at incredibly low prices

400+ fresh books written in 2025-2026 with today’s latest AI, Python, Machine Learning & tech trends – nowhere else will you find this complete country-wise collection on one platform!

Choose your country below and start reading instantly 🚀

BOOK LIBRARY USA 2026 LINK

BOOK LIBRARY INDIA 2026 LINK

BOOK LIBRARY AUSTRALIA 2026 LINK

BOOK LIBRARY CANADA 2026 LINK

BOOK LIBRARY UNITED KINGDOM 2026 LINK

BOOK LIBRARY GERMANY 2026 LINK

BOOK LIBRARY FRANCE 2026 LINK

BOOK LIBRARY ITALY 2026 LINK

BOOK LIBRARY SPAIN 2026 LINK

BOOK LIBRARY NETHERLANDS 2026 LINK

BOOK LIBRARY BRAZIL 2026 LINK

BOOK LIBRARY MEXICO 2026 LINK

BOOK LIBRARY JAPAN 2026 LINK

BOOK LIBRARY POLAND 2026 LINK

BOOK LIBRARY IRELAND 2026 LINK

BOOK LIBRARY SWEDEN 2026 LINK

BOOK LIBRARY BELGIUM 2026 LINK