e64d48a8c5
readme: updated logo
4 years ago
2fc3a1c38f
database: included sqlite and parameter substitution
...
Before we used insecure python variables to assemble a query string.
Now we use the DB-APIs parameter subtitution.
pymysqlite was switched for sqlite due to being included in python.
4 years ago
371c4f5064
Delete studip_crawler.png
4 years ago
85490cb616
logo: resize logo
4 years ago
4a8cf45ad3
database: switching from mysql to sqlite
...
Currently we only need the db to keep track of the files that were
already downloaded, thus we only use one database and only one table.
A complete sql database is a bit too bulky for this purpose.
By using sqlite we can minimize and embed the db.
4 years ago
415a21da3b
Update README.md
4 years ago
9a0dbc2feb
bugfixes: events without data-folder error
...
If an event didn't have a folder for files it was erroring.
This is fixed now by adding try-except blocks.
4 years ago
e8a0fe1000
bugfixes: wrong tag and repository name in docker-compose
4 years ago
d3c6d50803
bugfixes: added entry for workaround
4 years ago
3d8d71c3fd
bugfixes: included error message for large files - skipping them
4 years ago
9e6355f7de
bugfixes: keep searching for error with unwritable files
4 years ago
961ab04f5c
bugfixes: keep searching for error with unwritable files
4 years ago
818ad19329
bugfixes: error solving for unwritable files
4 years ago
b7b4683e0b
bugfixes: error solving for unwritable files
4 years ago
9d08df462b
log: show which courses will be downloaded
4 years ago
92ee51e56c
bugfixes: now all courses in the curr semester get downloaded
...
- limit for courses was set to low to display current courses
- unused variable docs was removed from local function
4 years ago
719c89e7cf
log: added logging for jobs to finish
4 years ago
bd4c9ab497
docker: fixed error in docker-compose.yml template
...
the username of my dockerhub account was misspelled
4 years ago
5d5d048790
"logo: readme underscore for correct link to logo"
4 years ago
96743fef36
"logo: fixed readme for correct logo"
4 years ago
889a883fcd
Merge branch 'master' of ssh://github/tiyn/stud.ip-crawler
4 years ago
d5d259d30a
"logo: added logo to readme"
4 years ago
ccc91e0fb1
Update README.md
4 years ago
4eb313eb9a
docker: added build and docker-compose
4 years ago
564a9ba977
temporarily removed the progressbar due to problems with logging
4 years ago
6f9ef9bbd6
Documentation: added docstrings
4 years ago
a5d7b0304b
merge
5 years ago
7ef3f063d2
log: adding basic log with levels and file
...
The log has options for several levels that can
be set from the command line.
The file is hardcoded as log.txt and can be toggled
5 years ago
edbda18180
Update README.md
5 years ago
302d135c94
Update README.md
5 years ago
fa36e0f29e
database: files id and chdates are stored
...
- mysql creates database and tables to given mysql if not existent already
- mysql reads last change values from db
- mysql saves ch_date after downloading
- run now takes care for the variables of mysql and studip
5 years ago
6d18baa8b6
adding license
5 years ago
78dcc01112
update readme
5 years ago
704d4c86eb
initial commit
5 years ago
TiynGER
Marten
TiynGER