datadog

Instruction file imported from bradcstevens/github-copilot-agent-mode-starter-kit (.github/instructions/technologies/datadog.instructions.md). Copyright stays with the author.

• Coding Standards

  • Use black for code formatting to ensure consistent style.

  • Use isort to sort imports lexicographically.

  • Employ flake8 for style checks and to maintain code quality.

  • Utilize bugbear plugin for flake8 to identify potential bugs and design issues. Enable the following checks:

    • B001: Avoid bare except: clauses; prefer except Exception:. Catches unexpected events.
    • B003: Avoid direct assignment to os.environ. Use os.environ.clear() or env= argument to Popen.
    • B006: Do not use mutable data structures for argument defaults.
    • B007: Loop control variable should be used within the loop body.
    • B301: Use Python 3 .iter* methods on dictionaries.
    • B305: Use next() builtin instead of .next().
    • B306: Use str(e) or e.args to access exception messages.
    • B902: Use self for instance methods, and cls for class methods.

  • Enforce consistent logging format with logging-format flake8 plugin. Enable the following checks:

    • G001: Logging statements should not use string.format().
    • G002: Logging statements should not use % formatting.
    • G003: Logging statements should not use + concatenation.
    • G004: Logging statements should not use f-strings (Python 3.6+).
    • G010: Logging statements should not use warn; use warning instead.
    • G100: Logging statements should not use extra arguments unless whitelisted.
    • G201: Logging statements should not use error(..., exc_info=True); use exception(...) instead.
    • G202: Logging statements should not use redundant exc_info=True in exception.

  • Use type checking with mypy for improved code quality. Configure mypy in hatch.toml: toml [env.collectors.datadog-checks] check-types: true mypy-args = [ "--py2", "--install-types", "--non-interactive", "datadog_checks/", "tests/", ] mypy-deps = [ "types-mock==0.1.5", ]

  • Consider using flags like --check-untyped-defs and --disallow-untyped-defs for stricter type checking. Configure the files to be checked with mypy.ini file.

• Metric and Tag Naming Conventions

  • Use descriptive and meaningful names for metrics and tags.
  • Avoid abbreviations that might have multiple meanings.
  • Maintain consistency in naming across all teams, apps, and services.
  • Avoid reserved keywords.
  • Prefix metrics with a namespace depicting the application or service generating the data (e.g., http.nginx.response_time).
  • Metric names must start with a letter.
  • Metric names can only contain ASCII alphanumerics, underscores, and periods. Other characters are converted to underscores.
  • Metric names should not exceed 200 characters (preferably less than 100).
  • Tag names must start with a letter.
  • Tag names may contain alphanumerics, underscores, minuses, colons, periods, and slashes. Other characters are converted to underscores.
  • Tags can be up to 200 characters long (including both key and value) and support Unicode, but are converted to lowercase.
  • Use the key:value syntax for tags for optimal functionality. Commonly used metric tag keys are instance, name, and role.
  • Implement unified service tagging using env, service, and version tags.

• Effective Dashboard Design

  • Design dashboards carefully for effective visualization and correlation.
  • Utilize out-of-the-box dashboards provided by Datadog.
  • Contribute to community-curated dashboards to enhance visibility and correlation of observability data.
  • Follow integration dashboard best practices when contributing.
  • Consult the "Effective Dashboards" repository for dashboard design guidelines.

• Code Organization and Structure

  • Directory Structure:
    • Organize code into modules based on functionality (e.g., metrics, logs, tracing).
    • Use a common directory for shared utilities and helper functions.
  • File Naming Conventions:
    • Use descriptive file names that reflect the module's purpose (e.g., metrics_collector.py, log_processor.py).
  • Module Organization:
    • Keep modules small and focused on a single responsibility.
    • Use __init__.py files to define package structure and expose necessary functionality.
  • Component Architecture:
    • Design components with clear interfaces and separation of concerns.
    • Use dependency injection to manage dependencies between components.
  • Code Splitting:
    • Consider splitting large modules into smaller files to improve maintainability.
    • Use lazy loading for infrequently used modules.

• Common Patterns and Anti-patterns

  • Design Patterns:
    • Use the Observer pattern for subscribing to Datadog events.
    • Implement the Decorator pattern for adding custom functionality to Datadog integrations.
  • Recommended Approaches:
    • Use Datadog's API client libraries for interacting with the Datadog API.
    • Implement custom checks using Datadog's check framework.
  • Anti-patterns:
    • Avoid hardcoding API keys or other sensitive information in the code.
    • Avoid excessive logging, which can impact performance.
  • State Management:
    • Use appropriate data structures for storing state (e.g., dictionaries, lists).
    • Consider using a dedicated state management library for complex state management needs.
  • Error Handling:
    • Implement robust error handling to prevent application crashes.
    • Use try-except blocks to catch and handle exceptions.
    • Log errors to Datadog for monitoring and analysis.

• Performance Considerations

  • Optimization Techniques:
    • Use efficient data structures and algorithms.
    • Minimize network requests to the Datadog API.
    • Use caching to reduce database load.
  • Memory Management:
    • Avoid memory leaks by properly releasing resources.
    • Use garbage collection to clean up unused objects.
  • Bundle Size Optimization:
    • Remove unused dependencies from the project.
    • Use code minification to reduce bundle size.
  • Lazy Loading:
    • Load infrequently used modules or components on demand.

• Security Best Practices

  • Vulnerabilities:
    • Prevent injection attacks by validating user inputs.
    • Protect against cross-site scripting (XSS) attacks by encoding outputs.
  • Input Validation:
    • Validate all user inputs to prevent malicious data from entering the system.
    • Use regular expressions or other validation techniques to enforce input constraints.
  • Authentication and Authorization:
    • Use strong authentication mechanisms to verify user identities.
    • Implement authorization policies to control access to resources.
  • Data Protection:
    • Encrypt sensitive data at rest and in transit.
    • Use secure communication protocols (e.g., HTTPS) for API communication.
  • Secure API Communication:
    • Use TLS/SSL for secure communication with the Datadog API.
    • Store API keys securely and do not expose them in the code.

• Testing Approaches

  • Unit Testing:
    • Write unit tests for individual components to verify their functionality.
    • Use mocking to isolate components from external dependencies.
  • Integration Testing:
    • Write integration tests to verify interactions between components.
    • Use a testing framework like pytest for writing and running tests.
  • End-to-end Testing:
    • Write end-to-end tests to verify the entire application workflow.
    • Use a tool like Selenium or Cypress for automating browser tests.
  • Test Organization:
    • Organize tests into separate directories based on component or functionality.
    • Use descriptive test names to clearly indicate the purpose of each test.
  • Mocking and Stubbing:
    • Use mocking to simulate external dependencies during testing.
    • Use stubbing to replace complex components with simplified versions.

• Common Pitfalls and Gotchas

  • Frequent Mistakes:
    • Incorrectly configuring Datadog integrations.
    • Not properly handling errors or exceptions.
    • Overusing logging, which can impact performance.
  • Edge Cases:
    • Handling large volumes of data.
    • Dealing with intermittent network connectivity issues.
  • Version-specific Issues:
    • Be aware of compatibility issues between different versions of Datadog and its dependencies.
  • Compatibility Concerns:
    • Ensure compatibility between Datadog and other technologies used in the project.
  • Debugging Strategies:
    • Use Datadog's monitoring tools to track down performance bottlenecks and errors.
    • Use logging to trace the execution flow of the application.

• Tooling and Environment

  • Recommended Tools:
    • Use an IDE like VS Code or PyCharm for development.
    • Use a version control system like Git for managing code.
    • Use a build tool like Make or Poetry for building and deploying the application.
  • Build Configuration:
    • Use a build tool to automate the build process.
    • Configure the build tool to generate optimized bundles.
  • Linting and Formatting:
    • Use a linter like Flake8 to enforce coding standards.
    • Use a formatter like Black to automatically format code.
  • Deployment:
    • Use a deployment tool like Docker or Kubernetes for deploying the application.
    • Use a CI/CD pipeline to automate the deployment process.
  • CI/CD Integration:
    • Integrate Datadog into the CI/CD pipeline to automatically monitor and analyze application performance.
    • Use Datadog's API to create custom dashboards and alerts.