1
0
mirror of https://github.com/tiyn/stud.ip-crawler.git synced 2026-02-22 06:34:48 +01:00

15 Commits

Author SHA1 Message Date
TiynGER
2fc3a1c38f database: included sqlite and parameter substitution
Before we used insecure python variables to assemble a query string.
Now we use the DB-APIs parameter subtitution.
pymysqlite was switched for sqlite due to being included in python.
2020-11-17 12:48:46 +01:00
TiynGER
4a8cf45ad3 database: switching from mysql to sqlite
Currently we only need the db to keep track of the files that were
already downloaded, thus we only use one database and only one table.
A complete sql database is a bit too bulky for this purpose.
By using sqlite we can minimize and embed the db.
2020-11-17 11:04:03 +01:00
TiynGER
9a0dbc2feb bugfixes: events without data-folder error
If an event didn't have a folder for files it was erroring.
This is fixed now by adding try-except blocks.
2020-11-11 16:47:47 +01:00
TiynGER
3d8d71c3fd bugfixes: included error message for large files - skipping them 2020-11-11 00:45:30 +01:00
TiynGER
9e6355f7de bugfixes: keep searching for error with unwritable files 2020-11-10 21:10:34 +01:00
TiynGER
961ab04f5c bugfixes: keep searching for error with unwritable files 2020-11-10 20:24:46 +01:00
TiynGER
818ad19329 bugfixes: error solving for unwritable files 2020-11-10 15:17:28 +01:00
TiynGER
b7b4683e0b bugfixes: error solving for unwritable files 2020-11-10 14:52:40 +01:00
TiynGER
9d08df462b log: show which courses will be downloaded 2020-11-10 12:44:31 +01:00
TiynGER
92ee51e56c bugfixes: now all courses in the curr semester get downloaded
- limit for courses was set to low to display current courses
- unused variable docs was removed from local function
2020-11-10 11:51:56 +01:00
TiynGER
719c89e7cf log: added logging for jobs to finish 2020-11-10 11:19:24 +01:00
TiynGER
564a9ba977 temporarily removed the progressbar due to problems with logging 2020-11-08 16:41:30 +01:00
TiynGER
6f9ef9bbd6 Documentation: added docstrings 2020-10-25 16:22:02 +01:00
TiynGER
7ef3f063d2 log: adding basic log with levels and file
The log has options for several levels that can
be set from the command line.
The file is hardcoded as log.txt and can be toggled
2020-06-10 00:24:01 +02:00
TiynGER
fa36e0f29e database: files id and chdates are stored
- mysql creates database and tables to given mysql if not existent already
- mysql reads last change values from db
- mysql saves ch_date after downloading
- run now takes care for the variables of mysql and studip
2020-06-07 12:49:08 +02:00