myphs
0a5cada92d
FIX: Skip dirs without id or name instead of aborting
...
There are directories now that contain neither a name or an id
delivered by the API for some reason, looking like this:
{
'is_visible': True,
'is_readable': False,
'is_writable': True
}
which makes no sense at all. If such a directory is given the
processing is stopped here and all further directories are ignored.
This should not be the case.
This commit fixes this behavior, so that all directories are processed
as expected.
2022-05-21 16:52:21 +02:00
TiynGER
e64d48a8c5
readme: updated logo
2020-12-14 23:55:42 +01:00
TiynGER
2fc3a1c38f
database: included sqlite and parameter substitution
...
Before we used insecure python variables to assemble a query string.
Now we use the DB-APIs parameter subtitution.
pymysqlite was switched for sqlite due to being included in python.
2020-11-17 12:48:46 +01:00
Marten
371c4f5064
Delete studip_crawler.png
2020-11-17 12:05:52 +01:00
TiynGER
85490cb616
logo: resize logo
2020-11-17 12:05:18 +01:00
TiynGER
4a8cf45ad3
database: switching from mysql to sqlite
...
Currently we only need the db to keep track of the files that were
already downloaded, thus we only use one database and only one table.
A complete sql database is a bit too bulky for this purpose.
By using sqlite we can minimize and embed the db.
2020-11-17 11:04:03 +01:00
Marten
415a21da3b
Update README.md
2020-11-12 00:51:04 +01:00
TiynGER
9a0dbc2feb
bugfixes: events without data-folder error
...
If an event didn't have a folder for files it was erroring.
This is fixed now by adding try-except blocks.
2020-11-11 16:47:47 +01:00
TiynGER
e8a0fe1000
bugfixes: wrong tag and repository name in docker-compose
2020-11-11 00:55:49 +01:00
TiynGER
d3c6d50803
bugfixes: added entry for workaround
2020-11-11 00:49:17 +01:00
TiynGER
3d8d71c3fd
bugfixes: included error message for large files - skipping them
2020-11-11 00:45:30 +01:00
TiynGER
9e6355f7de
bugfixes: keep searching for error with unwritable files
2020-11-10 21:10:34 +01:00
TiynGER
961ab04f5c
bugfixes: keep searching for error with unwritable files
2020-11-10 20:24:46 +01:00
TiynGER
818ad19329
bugfixes: error solving for unwritable files
2020-11-10 15:17:28 +01:00
TiynGER
b7b4683e0b
bugfixes: error solving for unwritable files
2020-11-10 14:52:40 +01:00
TiynGER
9d08df462b
log: show which courses will be downloaded
2020-11-10 12:44:31 +01:00
TiynGER
92ee51e56c
bugfixes: now all courses in the curr semester get downloaded
...
- limit for courses was set to low to display current courses
- unused variable docs was removed from local function
2020-11-10 11:51:56 +01:00
TiynGER
719c89e7cf
log: added logging for jobs to finish
2020-11-10 11:19:24 +01:00
Marten
bd4c9ab497
docker: fixed error in docker-compose.yml template
...
the username of my dockerhub account was misspelled
2020-11-10 09:13:12 +01:00
TiynGER
5d5d048790
"logo: readme underscore for correct link to logo"
2020-11-10 08:51:21 +01:00
TiynGER
96743fef36
"logo: fixed readme for correct logo"
2020-11-10 08:50:02 +01:00
TiynGER
889a883fcd
Merge branch 'master' of ssh://github/tiyn/stud.ip-crawler
2020-11-10 08:49:22 +01:00
TiynGER
d5d259d30a
"logo: added logo to readme"
2020-11-10 08:48:01 +01:00
Marten
ccc91e0fb1
Update README.md
2020-11-09 23:02:23 +01:00
TiynGER
4eb313eb9a
docker: added build and docker-compose
2020-11-09 22:46:21 +01:00
TiynGER
564a9ba977
temporarily removed the progressbar due to problems with logging
2020-11-08 16:41:30 +01:00
TiynGER
6f9ef9bbd6
Documentation: added docstrings
2020-10-25 16:22:02 +01:00
TiynGER
a5d7b0304b
merge
2020-06-10 00:24:28 +02:00
TiynGER
7ef3f063d2
log: adding basic log with levels and file
...
The log has options for several levels that can
be set from the command line.
The file is hardcoded as log.txt and can be toggled
2020-06-10 00:24:01 +02:00
Marten
edbda18180
Update README.md
2020-06-09 15:00:44 +02:00
Marten
302d135c94
Update README.md
2020-06-07 10:55:41 +00:00
TiynGER
fa36e0f29e
database: files id and chdates are stored
...
- mysql creates database and tables to given mysql if not existent already
- mysql reads last change values from db
- mysql saves ch_date after downloading
- run now takes care for the variables of mysql and studip
2020-06-07 12:49:08 +02:00
TiynGER
6d18baa8b6
adding license
2020-06-05 20:03:47 +02:00
TiynGER
78dcc01112
update readme
2020-06-05 19:48:26 +02:00
TiynGER
704d4c86eb
initial commit
2020-06-05 19:47:36 +02:00
