Skip to content

Data Scraping Application 🕸️

Overview

The Data Scraping Application is a collaborative project developed by students between Hochschule Augsburg, Germany, and LNU University, Sweden, with the supervision of Softwerk AB company. This application allows users to scrape data from specific websites by inputting a URL, selecting scraping options, and retrieving structured data output. 📊

Project Features ✨

  • Web Scraping Functionality: Users can input a URL to scrape data.
  • Multiple Scraping Options: Select from various scraping methods to meet your needs.
  • User-Friendly Interface: Modern and user-friendly front-end built using React.js.

  • Data Processing: Possible implementation of language models to summarize and clean scraped data.

  • Scrapes static websites using Requests and BeautifulSoup

  • Scrapes dynamic websites using Selenium
  • Option to clean scraped data
  • Preview or save the scraped data as a .txt file
  • Reliable and secure REST API with JWT Token

Tech Stack 🛠️

  • Front-End: React.js, Tailwind CSS
  • Back-End: Flask, SQLite, SQLAlchemy
  • Scraping Libraries: Requests, Beautifulsoup4, Selenium

Installation ⚡

To get started with the Data Scraping Application, clone the repository and install the required dependencies. 

git clone https://github.com/your-username/data-scraping-application.git
cd data-scraping-application


To run frontend and backend locally, please refer to our developer documentation. You can navigate to the relevant pages using the top navbar.

Enjoy Scrapping
(❁´◡`❁)