Almost Wrong Way To Send Docker Containers logs To ELK

In this article, we’ll walk through setting up a Docker-based ELK (Elasticsearch, Logstash, and Kibana) stack to collect, view, and send Docker logs. services: elasticsearch: image: elasticsearch:7.17.24 environment: - discovery.type=single-node volumes: - ./elasticsearch_data/:/usr/share/elasticsearch/data mem_limit: "1g" redis-cache: image: redis:7.4.0 logstash-agent: image: logstash:7.17.24 volumes: - ./logstash-agent:/etc/logstash command: logstash -f /etc/logstash/logstash.conf depends_on: - elasticsearch ports: - 12201:12201/udp logstash-central: image: logstash:7.17.24 volumes: - ./logstash-central:/etc/logstash command: logstash -f /etc/logstash/logstash.conf depends_on: - elasticsearch kibana: image: kibana:7.17.24 ports: - 5601:5601 environment: - ELASTICSEARCH_HOSTS=http://elasticsearch:9200 depends_on: - elasticsearch ElasticSearch Just create a folder named elasticsearch_data for storing data....

October 16, 2024 · 3 min · Alex Popescu

Backup PostgreSQL with Docker in S3 Object Storage

How can you backup your PostgreSQL to a Hetzner S3 object storage ? Here is how it’s done. Creating S3 Object Storage First, let’s create our S3 Object Storage: Next, we should create our S3 access key and secret: Backup and restore Let’s create a database with sample data: -- Create the database CREATE DATABASE music_streaming_platform; \c music_streaming_platform; -- Create users table CREATE TABLE users ( user_id SERIAL PRIMARY KEY, username VARCHAR(50) UNIQUE NOT NULL, email VARCHAR(100) UNIQUE NOT NULL, date_joined DATE DEFAULT CURRENT_DATE, country VARCHAR(50) ); -- Create artists table CREATE TABLE artists ( artist_id SERIAL PRIMARY KEY, artist_name VARCHAR(100) UNIQUE NOT NULL, genre VARCHAR(50), country VARCHAR(50), active_since DATE ); -- Create albums table CREATE TABLE albums ( album_id SERIAL PRIMARY KEY, album_name VARCHAR(100) NOT NULL, release_date DATE, artist_id INT REFERENCES artists(artist_id) ON DELETE CASCADE, genre VARCHAR(50) ); -- Create songs table CREATE TABLE songs ( song_id SERIAL PRIMARY KEY, song_name VARCHAR(100) NOT NULL, duration TIME NOT NULL, album_id INT REFERENCES albums(album_id) ON DELETE CASCADE, plays INT DEFAULT 0, release_date DATE ); -- Create playlists table CREATE TABLE playlists ( playlist_id SERIAL PRIMARY KEY, playlist_name VARCHAR(100) NOT NULL, user_id INT REFERENCES users(user_id) ON DELETE CASCADE, created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP ); -- Create playlist_songs table (many-to-many relationship between playlists and songs) CREATE TABLE playlist_songs ( playlist_id INT REFERENCES playlists(playlist_id) ON DELETE CASCADE, song_id INT REFERENCES songs(song_id) ON DELETE CASCADE, PRIMARY KEY (playlist_id, song_id) ); -- Insert data into users table INSERT INTO users (username, email, country) VALUES ('musiclover_101', 'lover101@example....

October 12, 2024 · 5 min · Alex Popescu

Running a CLI Before Your Main App In Docker

Do you ever want to run a CLI before your main app in Docker? Here is a complicated way to do just that. We need a folder named success_flag that will contain a flag for the main app to know when the cli has finished running. First, we delete any existing success flag that remains: base: build: . container_name: base volumes: - ./success_flag:/success_flag command: bash -c "rm -f /success_flag/cli_success" restart: "no" Second, let’s run the database layer, in this case Postgres (but it can be any database engine)....

September 19, 2024 · 2 min · Alex Popescu

Environment variables at build time with Docker

Introduction Do you need a certain environment variable at build time in Docker as opposed to runtime? There is an easy way to achieve this. Use Docker build arguments in combination with environment variables. An example of this is below of a React JS docker build that needs the environment variable REACT_APP_BACKEND_API and build time in the command npm run build. # Declare the build argument in the global scope ARG REACT_APP_BACKEND_API_ARG=TEST # Use an official Node runtime as a parent image FROM node:20-alpine # Set the working directory WORKDIR /app # Copy package....

August 1, 2024 · 2 min · Alex Popescu

From Source to Container: Building Efficient Quarkus Docker Images

In this article, we continue the trend from the previous article here and we will explore a few ways of creating a Docker image for a Quarkus app. Minimal install docker file Now with Quarkus, there are two ways of compiling a application: the normal java way and the native way. Quarkus native images and Java application packages have some significant differences. Java application packages contain a compiled bytecode that requires the JVM to run, while Quarkus native images are pre-compiled and optimized for a specific platform (in this example Linux), resulting in a smaller size and faster startup time....

May 5, 2023 · 5 min · Alex Popescu