English | MP4 | AVC 1920×1080 | AAC 44KHz 2ch | 48 Lessons (10h 39m) | 2.98 GB
Master the art of extracting data from the web
I’ve made a living for 3+ years from just web scraping, made over $125k, have 25k+ social media followers, and in this course I’m going to reveal exactly how I scrape sites, and how to get clients/grow on social media.
Here’s what you’ll get:
In the course I show you how to scrape these sites (and give you the code):
- Tiktok
- Zillow
- Google Search
- Google Maps
- Amazon
- Indeed
- ESPN
- Angi List
- And more!
I’ll also show you how I have gotten clients, and how I’ve gotten my 25k+ followers on social media (and how you can too)
What’s Inside:
- 9 hours of videos
- Is web scraping legal?
- Scrapers vs Hackers
- All the services I use (proxies, coding language, servers, database, packages to get around getting blocked)
- How websites get their data (how API’s work)
- My process for scraping ANY site
- How to find where data is coming from
- How to avoid getting blocked
- All about proxies
- Scraping by web framework (React, Next.js, static HTML/server rendered, ASP.NET, Java)
- How to reverse engineer sites with good blocking
- Scraping sites with Cursor Based Pagination (YouTube comments)
- Scraping a chrome extension for free emails
- How to scrape mobile apps
- How to skip trace (find random people’s phone numbers and emails)
- How to get Puppeteer running on AWS Lambda
- How to intercept requests with Puppeteer
- Solving captchas with Puppeteer
- How to get web scraping clients
- How to charge clients
- Product ideas that rely on web scraping
Table of Contents
1 Is Web Scraping Legal
2 Hackers vs Scrapers
3 My go to packages, editor, language, proxies, etc
4 How websites get their datahow API’s work
5 My Process for Scraping Any Site
6 How to find where data is coming from
7 How to set up proxies
8 How to make money web scraping
9 Scraping Next.js sites (angi, imdb)
10 Zillow – Scraping Hidden API’s
11 ESPN – Scraping API’s
12 Amazon – Straight up HTML page
13 Indeed – Straight Up HTML
14 Sarpy – ASP.NET (long version)
15 Sarpy – ASP.NET Cliff Notes
16 Kitsap – Java Site
17 Google Search
18 Google Maps
19 Linkedin
20 Shopify
21 They can’t hide anything from you….
22 Reverse Engineering Realtor API
23 Lynchburg exposing Auth Token
24 Beacons.ai
25 Twitter
26 Instagram
27 Arcgis
28 Cursor Based Pagination (YouTube Comments)
29 How to scrape Chrome Extensions (free emails)
30 Reverse Engineering a Private API with mitmproxy
31 Scraping Mobile Apps
32 Skip Tracing
33 Getting Puppeteer Up and Running on AWS Lambda
34 Getting Tiktok Videos with Puppeteer
35 Solving CAPTCHAs with Puppeteer
36 Sitemaps (crunchbase)
37 Google Tricks
38 Decoding Imperva Security Measures
39 Leveraging Google’s Web Cache to get around Cloudflare and Datadome
40 Tiktok API
41 SC Courts, use undetected chrome driver
42 Exploring Supabase for Efficient Database Management
43 Where to host the scrapers Render.com
44 Quick note on Concurrency in Web Scraping
45 How to get web scraping clients
46 What do you charge clients
47 What should you scrape for
48 I found a way to scrape Ohio State Student Emails
Resolve the captcha to access the links!