Course Information
Registration period
Year-round Recruitment
Course Level
Study Mode
Duration
2 Hour(s) 14 Minute(s)
Language
English
Taught by
Patrick Schroeder
Rating
3 views
Course Overview
Become a better JavaScript Developer and Learn Front-End Testing. We'll use javascript, lodash and jquery for scraping.
In this course you will learn how to scrape data from web pages using CasperJS.
This course consists of 5 example projects to help you fully understand the powers of the headless browser using the CasperJS API.
What You Will Learn
You will gain a thorough understanding of advanced web scraping concepts and also gain an insight into how to use the CasperJS for Testing DOM manipulation and UI interaction.
What to Expect
- We'll begin with an overview of how both PhantomJS and CasperJS works along with how to install these frameworks.
- Next, we'll discuss what our workflow will look like and the options we can pass into a Casper object.
- Then we'll dive into the meat of this course by working through 5 projects.
The Projects Will Cover
- How to scrape websites that are rendered with JavaScript instead of standard HTML
- How to wait for AJAX loaded data to appear before scraping elements
- How to submit forms both for Authorization and when making searches
- How to define navigation Steps - like logging into a site, clicking a button and following links
- How to write and save specified data in tables then output as an .html file or as JSON.
- And how to take screenshots both of full web pages and specific containers
What is PhantomJS?
PhantomJS is a Full Web Stack that employs a headless browser. Phantom gives us the power to perform many interesting actions on a web page, such as: performing page manipulation, simulating user interaction and the ability to dynamically capture and save website data.
What is CasperJS?
CasperJS is a stand-alone framework built on top Phantom and is compatible with most operating systems. The focus of this course will be on the Casper API and we'll be using this API to write all our web scraping scripts.
What You Should Know
You should already know JavaScript basics. Including what a callback function is. It will help if you know some jQuery. We use lodash in of our examples but only as a replacement for the built-in Map method that's part of the native Javascript API.
Course Content
- 9 section(s)
- 32 lecture(s)
- Section 1 Introduction
- Section 2 Overview and Install
- Section 3 Getting Started with CasperJS
- Section 4 Scraping Search Results
- Section 5 Scraping JavaScript Rendered Web Pages
- Section 6 Scraping Hotel Data
- Section 7 Scrape and Capture Multiple Pages
- Section 8 Log In and Search
- Section 9 Conclusion
What You’ll Learn
- Know how to use JavaScript for Data Mining
- Be able to Capture, Download and Save Website Data
- Understand how to use CasperJS and PhantomJS
- Apply What You've Learned to Front-end Testing
- Create Your Own Scripts for Scraping Data
- Have a Better Understanding of Functional Programming
- Fully Understand JavaScript and jQuery Selectors
Reviews
-
RRuan Osório
Este curso fraco, longe do esperado. Alguns dos códigos-fonte postados ao lado dos vídeos não funcionam, pois os websites passaram por atualizações. Didática do instrutor é bem ruim, pois os casos de uso utilizados para a exploração da ferramenta são maçantes o que torna o curso fraco.
-
AAnonymized User
Great quality tutorial, can breeze through it and get a grasp of a load of fundamentals quickly, enough for me to understand and apply to my own use case easily. PhantomJS coupled with CasperJS makes scraping, or browser testing really easy to work with, it's easy to write very minimal, clean code, even for front end coders like me, or even non-coders by using the jQuery API. Now, writing powerful server side code is precisely as easy as writing any other jQuery. Awesome tech, awesome tutorial. Money well spent to rapidly skill up and progress my app project in hours. Thanks! I also quite like the payment structure of Udemy, pay-per-course and access them forever. Ideal for those who may only be able to commit less time to learning, where a higher fixed monthly cost maybe more expensive - unless you know you can get through several courses per month, even then, I would be surprised to find higher quality, or better value courses elsewhere.
-
GGaurav Tripathi
Its good and have clearly explain real world web scrapping but would like to know scrapping data has to be updated into Database also instead of screen capturing into a image , would like to store directly it in database too.
-
RRobinson Marquez
This will definitively improve my career and help to use it in my job. There is actually a project coming up that involve scraping.
