How to Extract Data from News Articles and News Websites?

The main benefit of scraping articles and news websites data is that you can do it virtually from any website — as long as the content is online, it is possible to scrape it, starting from weather forecasts to government spending, if the particular website does not have an API for raw data access.

June 10, 2021

Our achievements in the field of business digital transformation.

Easy and Free Web Scraping

For the given project, we will utilize 3i Data Scraping scraper, a powerful data scraper, which can scrape data from all websites. You can download and install it now.

A web data scraper will permit you to extract website data you want to scrape as well as click on data that you wish to extract. Then, the scraper will automate the procedure and extract data on to the excel spreadsheet. In this example, we will extract the news feeds pages from Newsweek website.

Scraping News Article Data

Let’s start the data scraping project.

Ensure to download as well as install 3i Data Scraping scraper before you start.

1. Open 3i Data Scraping scraper and click on the “New Project”. Provide the URL that you wish to extract and we would submit Newsweek URL that we have chosen. 3i Data Scraping will render the site within the app.

2. Begin by clicking the title about the initial news article available on a page. This will be underlined in green and indicate that this has been chosen.

3. The rest headlines on a page would get highlighted in the yellow color. Then click on second one given on the page and choose them. They will now get highlighted with green color. On the left hand sidebar, rename the selection with headline.

4. After that, click on PLUS (+) symbol given next to your selected headline and select the command, “relative select”.

5. Then, use the command ‘Relative Select’ and click on Headline of first article as well as then on a category given above that. One arrow will come to show an association that you’re making. Rename the selection with category.

Repeat the steps 4-5 for adding an article’s byline. The project will now look like:

Need to find out how to extract more data? Then check our in-depth guide about how to extract data from a website.

Adding Pagination

Now, 3i Data Scraping is scraping the data that you’ve chosen from the initial page of different news articles. Now, we will tell 3i Data Scraping to extract extra article pages.

1. After that, click on PLUS (+) symbol next to the page selection as well as pick Select command.

2. Then scroll down to bottom of a page as well as click on “next page” control. Rename the selection with next.

3, Utilize the icon subsequent to the next selection for expanding it.

4. Then, delete both the extractions with this command.

5. After that, click on PLUS (+) symbol next to the next command as well as pick the click command.

6. The pop-up will come and ask you if it is the next page’s link. Then, click on the “yes” button and enter total number of times you would want to repeat this procedure. Here, we would repeat that 5 or more times.

Run The Web Data Scraping Project

Now, it’s time to test the scraping project. For doing that, just click on a green button “Get Data” given on the left-hand sidebar.

There, you can test, run, as well as schedule the project. Here, we will run that straight away.

Now, 3i Data Scraping will go and get data that you’ve demanded from the site. When the scraping gets completed, you will get a notification.

Note: Remember that a few news websites may block some of your IPs for doing web scraping. To solve this, you may require to turn on the IP Rotation option in 3i Data Scraping scraper.

Final Thoughts

When your run gets completed, you would be able to download that in an Excel or CSV format or in a JSON file.

Now, you understand how to extract data from news websites. In case, you face any problems while setting your project up, contact us via live chat or fill up the form and we’ll happily assist you!