parent
c57354990b
commit
7f6bb4a6f5
@ -1,3 +1,47 @@
|
||||
# SpellScrapper
|
||||
# Spell Scrapper :scroll: :snake:
|
||||
|
||||
Spell scrapper in Python for SpellTastic.
|
||||
## Data source
|
||||
All data is retrieved from [d20pfsrd](https://www.d20pfsrd.com/) the __#1 Pathfinder Roleplaying Game rules reference site__. All spells can be found at [spells](https://www.d20pfsrd.com/magic/all-spells/).
|
||||
|
||||
The latest data extracted is available as a YAML file.
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
* Python 3.6+
|
||||
|
||||
#### Python libraries
|
||||
* BeatifulSoup4
|
||||
* Requests
|
||||
* lxml
|
||||
* PyYAML
|
||||
* sqlite3
|
||||
|
||||
### Installing
|
||||
1. Cloning repository
|
||||
```
|
||||
git clone https://github.com/your_username/pathfinder-spell-scraper.git
|
||||
```
|
||||
2. Install the required libraries
|
||||
```
|
||||
pip install requests beautifulsoup4 lxml pyyaml
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Scrapping
|
||||
3. You can run __scrap-spells.py__ to scrape the spell information from the website:
|
||||
```
|
||||
python3 scrap-spells.py
|
||||
```
|
||||
__The script should take a few minutes to scrap all spells__
|
||||
|
||||
4. This command will generate a file __spells.yaml__ with all spells and their attributes
|
||||
|
||||
### Database
|
||||
5. You can build a __.db__ sqlite3 databse file by running the __spell_db.py__ file:
|
||||
```
|
||||
python3 spell-db.py
|
||||
```
|
||||
|
||||
6. The script will create a __spells.db__ file with a spell table containing all the spell information.
|
||||
|
Loading…
Reference in new issue