ScrapeMaster 1.0 - Installation Guide

1️⃣ Create a Virtual Environment

Run the following command to create and activate a virtual environment:

python -m venv venv

2️⃣ Install Dependencies

Create a requirements.txt file and copy the following dependencies:

openai
python-dotenv
pandas
pydantic
requests
beautifulsoup4
html2text
tiktoken
selenium
readability-lxml
streamlit
streamlit-tags
openpyxl

Then install all dependencies with:

pip install -r requirements.txt

3️⃣ Add Your API Key

Create a .env file and add your OpenAI API key:

OPENAI_API_KEY=sk-xxxxxxxx(place your own key)

4️⃣ Download ChromeDriver

Download ChromeDriver from the official website:Chrome for Testing availability

5️⃣ Create the Scraper Script

Save the following script as scraper.py:

// Loading...

6️⃣ Create the Streamlit App

Save the following as streamlit_app.py:

// Loading...

7️⃣ Run the Streamlit App

Run the following command:

streamlit run streamlit_app.py