tiyn/wiki
1
0
Fork 0
mirror of https://github.com/tiyn/wiki.git synced 2026-05-01 15:41:35 +02:00
Code Issues Projects Releases Wiki Activity

Compare commits

base: tiyn:ede99678ea15815d0862d53549e65cdf36c10fc0
Branches Tags
tiyn:master
..
compare: tiyn:master
Branches Tags
tiyn:master

4 Commits
ede99678ea ... master

Author SHA1 Message Date
tiyn
f990fa6557 KohakuHub: Added and linked into git (general) 2026-04-26 08:37:57 +02:00
tiyn
456c436e09 hierarchical data format: fixed structure 2026-04-23 04:10:16 +02:00
tiyn
f2c8027c4d Merge branch 'master' of github.com:tiyn/wiki 2026-04-19 02:00:49 +02:00
tiyn
603cecf8c4 HDF: Added 2026-04-19 02:00:42 +02:00
5 changed files with 198 additions and 25 deletions
Expand all files Collapse all files

2
wiki/android.md
View File

@@ -11,7 +11,7 @@ A free and open-source alternative to it is [F-Droid](/wiki/android/f-droid.md)
open-source software.
A list of open-source software options is also available on Alberto Mosconis
[Github](/wiki/git_%28general%29.md#server-with-web-interface) repository
[Github](/wiki/git_%28general%29.md#server) repository
[foss apps](https://github.com/albertomosconi/foss-apps/tree/main).
Additionally, open-source apps can be installed from source using

143
wiki/docker/kohakuhub_-_kohakuhub.md Normal file
View File

@@ -0,0 +1,143 @@
# kohakuhub - kohakuhub
This is a [Docker](/wiki/docker.md) container for a
[KohakuHub](https://github.com/KohakuBlueleaf/KohakuHub) server.
The Server consists of multiple Docker containers.
These are containers for the KohakuHub API and - depending on configuration - MinIO, LakeFS,
[Postgres](/wiki/docker/postgres.md)
## Set-up
Create the files `rebuild.sh` and `docker-compose.yml` at the same place.
The file `docker-compose.yml` is usually created automatically.
For this first download the [repository](https://github.com/KohakuBlueleaf/KohakuHub).
Then run the following command to generate the `docker-compose.yml` file.
```sh
python scripts/generate_docker_compose.py
```
Alternatively the example file `docker-compose.example.yml` file can be copied.
An additional example can be found in the [corresponding section](#docker-composeyml).
Change the settings according to your needs and run `./rebuild.sh` afterwards.
To publish the page on a domain the nginx container usually called `hub-ui` has to be published -
for example using a reverse proxy like
[traefik](/wiki/traefik.md#reverse-proxies-for-docker-service).
### Environment Variables
Set the following environment-variables in the `environment:` section of the
docker-compose file.
There are many other environment variables, which are automatically generated and not listed here.
| Name | Usage | Default |
| ----------------------------------------- | ------------------------------------ | ------- |
| `KOHAKU_HUB_INVITATION_ONLY` | Will disable public registration | false |
### Volumes
There are some volumes, which are automatically generated and not listed here.
### Ports
There are no ports, which are exposed by default.
### rebuild.sh
```sh
#!/bin/sh
docker-compose down
docker-compose up -d
```
### docker-compose.yml
```yml
services:
hub-ui:
image: nginx:alpine
restart: always
volumes:
- ./src/kohaku-hub-ui/dist:/usr/share/nginx/html
- ./src/kohaku-hub-admin/dist:/usr/share/nginx/html-admin
- ./docker/nginx/default.conf:/etc/nginx/conf.d/default.conf
depends_on:
- hub-api
hub-api:
build: .
restart: always
depends_on:
- postgres
- lakefs
- minio
environment:
- KOHAKU_HUB_INVITATION_ONLY=true
- KOHAKU_HUB_BASE_URL=https://hub.sub.domain.tld
- KOHAKU_HUB_S3_PUBLIC_ENDPOINT=https://s3.sub.domain.tld
- KOHAKU_HUB_SESSION_SECRET=CHANGE_ME_SESSION_SECRET
- KOHAKU_HUB_ADMIN_SECRET_TOKEN=CHANGE_ME_ADMIN_TOKEN
- KOHAKU_HUB_DATABASE_KEY=CHANGE_ME_DATABASE_KEY
- KOHAKU_HUB_DB_BACKEND=postgres
- KOHAKU_HUB_DATABASE_URL=postgresql://user:password@postgres:5432/database
- KOHAKU_HUB_S3_ENDPOINT=http://minio:9000
- KOHAKU_HUB_S3_ACCESS_KEY=CHANGE_ME_S3_ACCESS_KEY
- KOHAKU_HUB_S3_SECRET_KEY=CHANGE_ME_S3_SECRET_KEY
- KOHAKU_HUB_S3_BUCKET=example-bucket
- KOHAKU_HUB_S3_REGION=us-east-1
- KOHAKU_HUB_LAKEFS_ENDPOINT=http://lakefs:28000
- KOHAKU_HUB_LAKEFS_REPO_NAMESPACE=example
- KOHAKU_HUB_SITE_NAME=ExampleHub
- KOHAKU_HUB_AUTO_MIGRATE=true
volumes:
- ./hub-meta/hub-api:/hub-api-creds
minio:
image: quay.io/minio/minio:latest
command: server /data --console-address ":29000"
restart: unless-stopped
environment:
- MINIO_ROOT_USER=CHANGE_ME_MINIO_USER
- MINIO_ROOT_PASSWORD=CHANGE_ME_MINIO_PASSWORD
volumes:
- ./hub-storage/minio-data:/data
- ./hub-meta/minio-data:/root/.minio
lakefs:
build:
context: ./docker/lakefs
entrypoint: ["/bin/sh", "/scripts/lakefs-entrypoint.sh"]
command: ["run"]
restart: unless-stopped
depends_on:
- minio
- postgres
environment:
- LAKEFS_DATABASE_TYPE=postgres
- LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING=postgres://user:password@postgres:5432/lakefs?sslmode=disable
- LAKEFS_BLOCKSTORE_TYPE=s3
- LAKEFS_BLOCKSTORE_S3_ENDPOINT=http://minio:9000
- LAKEFS_BLOCKSTORE_S3_BUCKET=example-bucket
- LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE=true
- LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=CHANGE_ME_S3_ACCESS_KEY
- LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY=CHANGE_ME_S3_SECRET_KEY
- LAKEFS_BLOCKSTORE_S3_REGION=us-east-1
- LAKEFS_AUTH_ENCRYPT_SECRET_KEY=CHANGE_ME_LAKEFS_SECRET
- LAKEFS_LISTEN_ADDRESS=0.0.0.0:28000
volumes:
- ./hub-meta/lakefs-data:/var/lakefs/data
- ./hub-meta/lakefs-cache:/lakefs/data/cache
- ./docker/lakefs/lakefs-entrypoint.sh:/scripts/lakefs-entrypoint.sh:ro
- ./docker/lakefs/init-databases.sh:/scripts/init-databases.sh:ro
postgres:
image: postgres:15
restart: always
environment:
- POSTGRES_USER=user
- POSTGRES_PASSWORD=password
- POSTGRES_DB=database
volumes:
- ./hub-meta/postgres-data:/var/lib/postgresql/data
```

51
wiki/git_(general).md
View File

@@ -16,36 +16,41 @@ has a [separate article](/wiki/linux/git_%28package%29.md).
### Server
The server hosts the data that is needed for the versioning.
A distinction can be made between
[GUI-less servers](#server-without-graphical-interface) and
[servers with a web-GUI](#server-with-web-interface).
There are many different [server](#server) which also have
[distinct applications](#distinct-git-server-apps) other than the normal [client](#client).
#### Server with web interface
#### Server
Web interfaces for git are quite useful for easily showing code to other people.
Additionally, it is easy to collaborate together.
- [Gitea](gitea.md) is an open-source and selfhosted git instance which can be dockerized.
- [Gitea](gitea.md) is an open-source and self-hosted git instance which can be
[dockerized](/wiki/docker/gitea_-_gitea.md).
It has a graphical user interface.
- [GitHub](https://github.com) is a popular git server.
It is not selfhosted and owned by microsoft ([cli-tool](https://github.com/github/hub)).
It is not self-hosted and owned by Microsoft.
It has a graphical user interface.
- [GitLab](https://gitlab.com) is a self-hostable git instance which also has a default version
available via their website.
#### Git Apps
Depending on the [specific server](#server-with-web-interface) used there are many different apps
that can be used.
- For [GitHub](https://github.com) there is app with the same name for
[Android](/wiki/android.md) devices and iOS devices.
- For [Gitea](/wiki/gitea.md) and Forgejo there is an app called [GitNex](https://gitnex.com) which
is available for [Android](/wiki/android.md) devices.
- For [GitLab](https://gitlab.com) there is an app called [LabNex](https://labnex.app) which is
available for [Android](/wiki/android.md) devices.
#### Server without graphical interface
If you don't need a bloated web-interface a more basic approach is more than enough.
It has a graphical user interface.
- [git-server-docker](/wiki/docker/jkarlos_-_git-server-docker.md) is a pretty
basic docker-container for git.
It does not have a graphical user interface.
If a solution for storing datasets or AI models is needed [KohakuHub](/wiki/kohakuhub.md) is
available and uses similar techniques of storage and versioning although it is not a Git server.
#### Distinct Git Server Apps
Depending on the [specific server](#server) used there are many different apps
that can be used.
The normal git [clients](#client) can be used for all of them.
- [GitHub](https://github.com) features various apps with the same name for
[Android](/wiki/android.md) and iOS devices.
- [hub](https://github.com/github/hub) is an official command-line tool for
[GitHub](https://github.com).
- [GitNex](https://gitnex.com) is an app for [Android](/wiki/android.md) devices which can access
[Gitea](/wiki/gitea.md) and Forgejo instances.
- [LabNex](https://labnex.app) is an app for [Android](/wiki/android.md) devices which can access
[GitLab](https://gitlab.com) instances.

14
wiki/hierarchical_data_format.md Normal file
View File

@@ -0,0 +1,14 @@
# Hierarchical Data Format
Hierarchical Data Format - or in short HDF - is a file format to organize data in a hierarchical
way.
## Viewer
There are various possible viewers for `.hdf` files.
The following list contains some of them.
- [HDFView](https://github.com/HDFGroup/hdfview) is a graphical viewer for `.hdf` files.
It also features the ability to plot data.
- [ViTables](https://github.com/uvemas/ViTables) is a graphical viewer for `.hdf` files.
Contrary to other viewers it does not feature plotting data and is relatively simple.

11
wiki/kohakuhub.md Normal file
View File

@@ -0,0 +1,11 @@
# KohakuHub
[KohakuHub](https://hub.kohaku-lab.org/) is a self-hostable platform for AI models and datasets for
AI models.
It can be used as an alternative to the very popular service called
[HuggingFace](https://huggingface.co/).
## Setup
The software can be setup via [Docker](/wiki/docker.md) with the
[kohakuhub image](/wiki/docker/kohakuhub_-_kohakuhub.md).