4.3

Release notes

3.0.1 

  • All

    • Replacing log4j logging framework with logback to nullify threat from CVE-2021-44228.
    • Removing legacy dependencies (sql-core, json-sql, kafka-connect-smt) to simplify dependency chain.
  • Cassandra Sink Connector

    • Fix for #636 - CassandraJsonWriter does not log the table name for InvalidQueryException
  • HBase Sink Connector

    • Fix for #753 - Race condition in StructFieldsExtractorBytes

3.0.0 

  • All

    • Move to KCQL 2.8.9
    • Change sys.errors to ConnectExceptions
    • Additional testing with TestContainers
    • Licence scan report and status
  • AWS S3 Sink Connector

    • S3 Source Offset Fix
    • Fix JSON & Text newline detection when running in certain Docker images
    • Byte handling fixes
    • Partitioning of nested data
    • Error handling and retry logic
    • Handle preCommit with null currentOffsets
    • Remove bucket validation on startup
    • Enabled simpler management of default flush values.
    • Local write mode - build locally, then ship
    • Deprecating old properties, however rewriting them to the new properties to ensure backwards compatibility.
    • Adding the capability to specify properties in yaml configuration
    • Rework exception handling. Refactoring errors to use Either[X,Y] return types where possible instead of throwing exceptions.
    • Ensuring task can be stopped gracefully if it has not been started yet
    • ContextReader testing and refactor
    • Adding a simple state model to the S3Writer to ensure that states and transitions are kept consistent. This can be improved in time.
  • AWS S3 Source Connector

    • Change order of match to avoid scala.MatchError
    • S3 Source rewritten to be more efficient and use the natural ordering of S3 keys
    • Region is necessary when using the AWS client
  • Cassandra Sink & Source Connectors

    • Add connection and read client timeout
  • FTP Connector

    • Support for Secure File Transfer Protocol
  • Hive Sink Connector

    • Array Support
    • Kerberos debug flag added
  • Influx DB Sink

    • Bump influxdb-java from version 2.9 to 2.29
    • Added array handling support
  • MongoDB Sink Connector

    • Nested Fields Support
  • Redis Sink Connector

    • Fix Redis Pubsub Writer
    • Add support for json and json with schema

2.1.3 

Move to connect-common 2.0.5 that adds complex type support to KCQL

2.1.2 

  • AWS S3 Sink Connector
    • Prevent null pointer exception in converters when maps are presented will null values
    • Offset reader optimisation to reduce S3 load
    • Ensuring that commit only occurs after the preconfigured time interval when using WITH_FLUSH_INTERVAL
  • AWS S3 Source Connector (New Connector)
  • Cassandra Source Connector
    • Add Bucket Timeseries Mode
    • Reduction of logging noise
    • Proper handling of uninitialized connections on task stop()
  • Elasticsearch Sink Connector
    • Update default port
  • Hive Sink
    • Improve Orc format handling
    • Fixing issues with partitioning by non-string keys
  • Hive Source
    • Ensuring newly written files can be read by the hive connector by introduction of a refresh frequency configuration option.
  • Redis Sink
    • Correct Redis writer initialisation

2.1.0 

  • AWS S3 Sink Connector
  • Elasticsearch 7 Support

2.0.1 

  • Hive Source

    • Rename option connect.hive.hive.metastore to connect.hive.metastore
    • Rename option connect.hive.hive.metastore.uris to connect.hive.metastore.uris
  • Fix Elastic start up NPE

  • Fix to correct batch size extraction from KCQL on Pulsar

2.0.0 

  • Move to Scala 2.12
  • Move to Kafka 2.4.1 and Confluent 5.4
  • Deprecated:
    • Druid Sink (not scala 2.12 compatible)
    • Elastic Sink (not scala 2.12 compatible)
    • Elastic5 Sink(not scala 2.12 compatible)
  • Redis
    • Add support for Redis Streams
  • Cassandra
    • Add support for setting the LoadBalancer policy on the Cassandra Sink
  • ReThinkDB
    • Use SSL connection on Rethink initialize tables is ssl set
  • FTP Source
    • Respect connect.ftp.max.poll.records when reading slices
  • MQTT Source
    • Allow lookup of avro schema files with wildcard subscriptions