Imported from apache/flink-cdc (AGENTS.md). Install upstream with npx skills add apache/flink-cdc. Copyright stays with the author.

Flink CDC AI Agent Instructions

This file provides guidance for AI coding agents working with the Apache Flink CDC.

Prerequisites

• Java 11 (baseline; all code must compile and run on Java 11) and Java 17
• Maven 3.8.6 or higher
• Git
• Unix-like environment (Linux, macOS, WSL)
• Docker (required for running integration tests and e2e tests)

Commands

Build

• Fast dev build (skip tests and format checks): mvn clean install -DskipTests -Dspotless.check.skip=true -Dcheckstyle.skip=true
• Full build against Flink 1.x (default): mvn clean package -DskipTests
• Full build against Flink 2.x: mvn clean package -DskipTests -Pflink2
• Single module (e.g. flink-cdc-common): mvn clean package -DskipTests -pl flink-cdc-common -am

Testing

• Run all tests for a module: mvn verify -pl flink-cdc-common
• Run all tests against Flink 2.x: mvn verify -pl flink-cdc-common -Pflink2
• Run a single test class: mvn -pl flink-cdc-common -Dtest=MyTest test
• Run a single test method: mvn -pl flink-cdc-common -Dtest=MyTest#myMethod test

Code Quality

• Before committing changes, run mvn spotless:apply and mvn spotless:check to enforce code style rules.
• Make sure newly added files have proper ASF license headers.

Repository Structure

Flink CDC is organized around the Pipeline abstraction: a user-defined data pipeline that reads from one or more sources, optionally transforms records, and writes to one or more sinks. The core modules implement this abstraction; the connector modules provide the concrete source and sink implementations.

Core Modules

• flink-cdc-common — Shared API and data model used across all modules: CDC event types (DataChangeEvent, SchemaChangeEvent, etc.), the schema model, data types, source/sink interfaces, Factory SPI, route definitions, UDF interfaces, and utility classes. Most new abstractions start here.
• flink-cdc-runtime — Runtime implementation of the Pipeline: operators for reading, routing, transforming (expression evaluation via Calcite + Janino), and writing CDC events.
• flink-cdc-composer — Pipeline assembly and deployment layer. Translates a PipelineDefinition into a runnable Flink job, wiring sources, operators, and sinks together. Supports Flink-native, Kubernetes, and YARN deployment.
• flink-cdc-cli — Command-line entry point (flink-cdc.sh). Parses YAML pipeline definitions and delegates to flink-cdc-composer.
• flink-cdc-dist — Distribution packaging. Produces the flink-cdc-<version>-bin release archive.

Flink Version Compatibility

Currently, Flink CDC supports two Flink generations simultaneously:

• flink-cdc-flink1-compat — Flink 1.x compatibility layer (currently 1.20.3). Default profile.
• flink-cdc-flink2-compat — Flink 2.x compatibility layer (currently 2.2.0). Activated via -Pflink2.

All modules that depend on Flink APIs must declare their Flink dependencies as provided and reference ${flink.version}, which is resolved by the active profile. Please verify your changes in both Flink 1.20 (LTS) and Flink 2.x (latest).

Connectors (flink-cdc-connect/)

Connectors are split into two categories:

• Source Connectors (flink-cdc-connect/flink-cdc-source-connectors/) are CDC sources for DataStream and Flink SQL jobs.
• Pipeline Connectors (flink-cdc-connect/flink-cdc-pipeline-connectors/) are Pipeline connectors for the YAML API.

Tests

Write unit tests and integration tests in the corresponding submodules. End-to-end tests are located in these modules:

• flink-cdc-e2e-tests/ — End-to-end tests parent module.
  • flink-cdc-e2e-utils — Shared test utilities (container management, assertions)
  • flink-cdc-source-e2e-tests — E2E tests for source connectors
  • flink-cdc-pipeline-e2e-tests — E2E tests for pipeline connectors

Documentation

• docs/ — Hugo-based documentation site. docs/content/ for English, docs/content.zh/ for Chinese. Update both when adding new features.

Coding Standards

• Format Java files with Spotless before every commit: mvn spotless:apply.
• Import order (enforced by Checkstyle): org.apache.flink.cdc → org.apache.flink → other third-party → javax → java. Static imports go last. No star imports.
• Forbidden imports (enforced by Checkstyle):
  • JUnit 4 (org.junit.* except org.junit.jupiter.*) — use JUnit 5 Jupiter instead
  • org.junit.jupiter.api.Assertions and org.hamcrest — use AssertJ instead
  • com.google.common.* — use flink-shaded-guava instead
  • com.google.common.base.Preconditions — use Flink CDC's Preconditions
  • com.google.common.annotations.VisibleForTesting — use @VisibleForTesting from org.apache.flink.cdc.common.annotation
• API stability annotations: User-facing API types should carry stability annotations, and methods/fields/constructors only need explicit annotations when they differ from the enclosing type or when needed.
  • @Public — stable across major versions
  • @PublicEvolving — may change in minor versions
  • @Experimental — may change at any time
  • @Internal — no stability guarantee; users should not depend on it
• Logging: Use SLF4J with parameterized placeholders (LOG.info("foo {}", bar)), never string concatenation.
• Braces: Always use braces for if/else/for/while/do blocks.
• Comments: All comments should be written in English, and keep it concise. Only write comments when necessary. Avoid trivial @param, @return comments in JavaDocs.
• Apache License 2.0 header required on all new files.

Testing Standards

• Use JUnit 5 (org.junit.jupiter) and AssertJ (org.assertj.core.api.Assertions) for all new tests. Do not use JUnit 4 or Hamcrest.
• Test classes are package-private (no public modifier on the class).
• Unit tests are named *Test.java (e.g., SchemaUtilsTest).
• Integration tests (requiring Docker / real databases) are named *ITCase.java (e.g., MySqlSourceITCase).
• Add tests for new behavior, covering success, failure, and edge cases. Tests should be self-explanatory; avoid verbose comments in test classes.
• For bug fixes, verify the new test fails without the fix before confirming it passes with it.

Commits and PRs

Commit message format

• [FLINK-XXXX][component] Description where FLINK-XXXX is the JIRA issue number and component is the affected area (e.g. connect/mysql, pipeline-connector/kafka, docs, runtime)
• [hotfix][component] ... or [docs][component] ... for trivial fixes without a JIRA issue
• [ci] Description for CI-only changes

Pull request conventions

• Title format mirrors the commit message: [FLINK-XXXX][component] Title
• A corresponding JIRA issue is required (except trivial changes)
• Fill out the PR template completely: describe purpose, change log, testing approach, and documentation impact
• Ensure CI passes before requesting review
• Enable GitHub Actions on your fork before opening a PR
• When AI tools were used, check the AI disclosure checkbox and uncomment the Generated-by line in the PR template, per ASF Generative Tooling Guidance

Boundaries

Ask first

• Adding or changing @Public or @PublicEvolving annotations (user-facing API commitments)
• New dependencies
• Large cross-module refactors
• Changes to serialization formats or checkpoint behavior
• Changes to hot paths that could impact performance (per-record processing, state access)

Never

• Commit secrets, credentials, or tokens
• Push directly to apache/flink-cdc; always work from your fork
• Mix unrelated changes into one PR
• Use JUnit 4 or Hamcrest in new test code
• Use org.junit.jupiter.api.Assertions; use AssertJ instead
• Add Co-Authored-By with an AI agent in commit messages; use Generated-by: <Tool Name and Version> instead