diff --git a/wiki/android.md b/wiki/android.md index e9957c0..fb6d684 100644 --- a/wiki/android.md +++ b/wiki/android.md @@ -11,7 +11,7 @@ A free and open-source alternative to it is [F-Droid](/wiki/android/f-droid.md) open-source software. A list of open-source software options is also available on Alberto Mosconis -[Github](/wiki/git_%28general%29.md#server-with-web-interface) repository +[Github](/wiki/git_%28general%29.md#server) repository [foss apps](https://github.com/albertomosconi/foss-apps/tree/main). Additionally, open-source apps can be installed from source using diff --git a/wiki/docker/kohakuhub_-_kohakuhub.md b/wiki/docker/kohakuhub_-_kohakuhub.md new file mode 100644 index 0000000..d3035f6 --- /dev/null +++ b/wiki/docker/kohakuhub_-_kohakuhub.md @@ -0,0 +1,143 @@ +# kohakuhub - kohakuhub + +This is a [Docker](/wiki/docker.md) container for a +[KohakuHub](https://github.com/KohakuBlueleaf/KohakuHub) server. +The Server consists of multiple Docker containers. +These are containers for the KohakuHub API and - depending on configuration - MinIO, LakeFS, +[Postgres](/wiki/docker/postgres.md) + +## Set-up + +Create the files `rebuild.sh` and `docker-compose.yml` at the same place. +The file `docker-compose.yml` is usually created automatically. +For this first download the [repository](https://github.com/KohakuBlueleaf/KohakuHub). +Then run the following command to generate the `docker-compose.yml` file. + +```sh +python scripts/generate_docker_compose.py +``` + +Alternatively the example file `docker-compose.example.yml` file can be copied. +An additional example can be found in the [corresponding section](#docker-composeyml). +Change the settings according to your needs and run `./rebuild.sh` afterwards. + +To publish the page on a domain the nginx container usually called `hub-ui` has to be published - +for example using a reverse proxy like +[traefik](/wiki/traefik.md#reverse-proxies-for-docker-service). + +### Environment Variables + +Set the following environment-variables in the `environment:` section of the +docker-compose file. +There are many other environment variables, which are automatically generated and not listed here. + +| Name | Usage | Default | +| ----------------------------------------- | ------------------------------------ | ------- | +| `KOHAKU_HUB_INVITATION_ONLY` | Will disable public registration | false | + +### Volumes + +There are some volumes, which are automatically generated and not listed here. + +### Ports + +There are no ports, which are exposed by default. + +### rebuild.sh + +```sh +#!/bin/sh +docker-compose down +docker-compose up -d +``` + +### docker-compose.yml + +```yml +services: + hub-ui: + image: nginx:alpine + restart: always + volumes: + - ./src/kohaku-hub-ui/dist:/usr/share/nginx/html + - ./src/kohaku-hub-admin/dist:/usr/share/nginx/html-admin + - ./docker/nginx/default.conf:/etc/nginx/conf.d/default.conf + depends_on: + - hub-api + + hub-api: + build: . + restart: always + depends_on: + - postgres + - lakefs + - minio + environment: + - KOHAKU_HUB_INVITATION_ONLY=true + - KOHAKU_HUB_BASE_URL=https://hub.sub.domain.tld + - KOHAKU_HUB_S3_PUBLIC_ENDPOINT=https://s3.sub.domain.tld + - KOHAKU_HUB_SESSION_SECRET=CHANGE_ME_SESSION_SECRET + - KOHAKU_HUB_ADMIN_SECRET_TOKEN=CHANGE_ME_ADMIN_TOKEN + - KOHAKU_HUB_DATABASE_KEY=CHANGE_ME_DATABASE_KEY + - KOHAKU_HUB_DB_BACKEND=postgres + - KOHAKU_HUB_DATABASE_URL=postgresql://user:password@postgres:5432/database + - KOHAKU_HUB_S3_ENDPOINT=http://minio:9000 + - KOHAKU_HUB_S3_ACCESS_KEY=CHANGE_ME_S3_ACCESS_KEY + - KOHAKU_HUB_S3_SECRET_KEY=CHANGE_ME_S3_SECRET_KEY + - KOHAKU_HUB_S3_BUCKET=example-bucket + - KOHAKU_HUB_S3_REGION=us-east-1 + - KOHAKU_HUB_LAKEFS_ENDPOINT=http://lakefs:28000 + - KOHAKU_HUB_LAKEFS_REPO_NAMESPACE=example + - KOHAKU_HUB_SITE_NAME=ExampleHub + - KOHAKU_HUB_AUTO_MIGRATE=true + volumes: + - ./hub-meta/hub-api:/hub-api-creds + + minio: + image: quay.io/minio/minio:latest + command: server /data --console-address ":29000" + restart: unless-stopped + environment: + - MINIO_ROOT_USER=CHANGE_ME_MINIO_USER + - MINIO_ROOT_PASSWORD=CHANGE_ME_MINIO_PASSWORD + volumes: + - ./hub-storage/minio-data:/data + - ./hub-meta/minio-data:/root/.minio + + lakefs: + build: + context: ./docker/lakefs + entrypoint: ["/bin/sh", "/scripts/lakefs-entrypoint.sh"] + command: ["run"] + restart: unless-stopped + depends_on: + - minio + - postgres + environment: + - LAKEFS_DATABASE_TYPE=postgres + - LAKEFS_DATABASE_POSTGRES_CONNECTION_STRING=postgres://user:password@postgres:5432/lakefs?sslmode=disable + - LAKEFS_BLOCKSTORE_TYPE=s3 + - LAKEFS_BLOCKSTORE_S3_ENDPOINT=http://minio:9000 + - LAKEFS_BLOCKSTORE_S3_BUCKET=example-bucket + - LAKEFS_BLOCKSTORE_S3_FORCE_PATH_STYLE=true + - LAKEFS_BLOCKSTORE_S3_CREDENTIALS_ACCESS_KEY_ID=CHANGE_ME_S3_ACCESS_KEY + - LAKEFS_BLOCKSTORE_S3_CREDENTIALS_SECRET_ACCESS_KEY=CHANGE_ME_S3_SECRET_KEY + - LAKEFS_BLOCKSTORE_S3_REGION=us-east-1 + - LAKEFS_AUTH_ENCRYPT_SECRET_KEY=CHANGE_ME_LAKEFS_SECRET + - LAKEFS_LISTEN_ADDRESS=0.0.0.0:28000 + volumes: + - ./hub-meta/lakefs-data:/var/lakefs/data + - ./hub-meta/lakefs-cache:/lakefs/data/cache + - ./docker/lakefs/lakefs-entrypoint.sh:/scripts/lakefs-entrypoint.sh:ro + - ./docker/lakefs/init-databases.sh:/scripts/init-databases.sh:ro + + postgres: + image: postgres:15 + restart: always + environment: + - POSTGRES_USER=user + - POSTGRES_PASSWORD=password + - POSTGRES_DB=database + volumes: + - ./hub-meta/postgres-data:/var/lib/postgresql/data +``` diff --git a/wiki/git_(general).md b/wiki/git_(general).md index cda5828..2d8d06a 100644 --- a/wiki/git_(general).md +++ b/wiki/git_(general).md @@ -16,36 +16,41 @@ has a [separate article](/wiki/linux/git_%28package%29.md). ### Server The server hosts the data that is needed for the versioning. -A distinction can be made between -[GUI-less servers](#server-without-graphical-interface) and -[servers with a web-GUI](#server-with-web-interface). +There are many different [server](#server) which also have +[distinct applications](#distinct-git-server-apps) other than the normal [client](#client). -#### Server with web interface +#### Server Web interfaces for git are quite useful for easily showing code to other people. Additionally, it is easy to collaborate together. -- [Gitea](gitea.md) is an open-source and selfhosted git instance which can be dockerized. -- [GitHub](https://github.com) is a popular git server. - It is not selfhosted and owned by microsoft ([cli-tool](https://github.com/github/hub)). +- [Gitea](gitea.md) is an open-source and self-hosted git instance which can be + [dockerized](/wiki/docker/gitea_-_gitea.md). + It has a graphical user interface. +- [GitHub](https://github.com) is a popular git server. + It is not self-hosted and owned by Microsoft. + It has a graphical user interface. - [GitLab](https://gitlab.com) is a self-hostable git instance which also has a default version available via their website. - -#### Git Apps - -Depending on the [specific server](#server-with-web-interface) used there are many different apps -that can be used. - -- For [GitHub](https://github.com) there is app with the same name for - [Android](/wiki/android.md) devices and iOS devices. -- For [Gitea](/wiki/gitea.md) and Forgejo there is an app called [GitNex](https://gitnex.com) which - is available for [Android](/wiki/android.md) devices. -- For [GitLab](https://gitlab.com) there is an app called [LabNex](https://labnex.app) which is - available for [Android](/wiki/android.md) devices. - -#### Server without graphical interface - -If you don't need a bloated web-interface a more basic approach is more than enough. - + It has a graphical user interface. - [git-server-docker](/wiki/docker/jkarlos_-_git-server-docker.md) is a pretty basic docker-container for git. + It does not have a graphical user interface. + +If a solution for storing datasets or AI models is needed [KohakuHub](/wiki/kohakuhub.md) is +available and uses similar techniques of storage and versioning although it is not a Git server. + +#### Distinct Git Server Apps + +Depending on the [specific server](#server) used there are many different apps +that can be used. +The normal git [clients](#client) can be used for all of them. + +- [GitHub](https://github.com) features various apps with the same name for + [Android](/wiki/android.md) and iOS devices. +- [hub](https://github.com/github/hub) is an official command-line tool for + [GitHub](https://github.com). +- [GitNex](https://gitnex.com) is an app for [Android](/wiki/android.md) devices which can access + [Gitea](/wiki/gitea.md) and Forgejo instances. +- [LabNex](https://labnex.app) is an app for [Android](/wiki/android.md) devices which can access + [GitLab](https://gitlab.com) instances. diff --git a/wiki/kohakuhub.md b/wiki/kohakuhub.md new file mode 100644 index 0000000..8ab2e3c --- /dev/null +++ b/wiki/kohakuhub.md @@ -0,0 +1,11 @@ +# KohakuHub + +[KohakuHub](https://hub.kohaku-lab.org/) is a self-hostable platform for AI models and datasets for +AI models. +It can be used as an alternative to the very popular service called +[HuggingFace](https://huggingface.co/). + +## Setup + +The software can be setup via [Docker](/wiki/docker.md) with the +[kohakuhub image](/wiki/docker/kohakuhub_-_kohakuhub.md).