目的
以下ブログを参考にKafkaのSnowflake Connectorを試したいです。
https://blog.devgenius.io/snowflake-connector-for-kafka-in-action-897da0e5f862
エラー
以下のエラーのとおり「ERROR Failed to create job for ./config/SF_connect.properties 」で、
SF_connect.propertiesの記述に問題があると推測しており、Snowflakeの以下ドキュメントを参考に何度も修正していますが、解決できない状況です。
https://docs.snowflake.com/ja/user-guide/kafka-connector-install#kafka-configuration-properties
[root@localhost kafka_2.12-3.4.0]# ./bin/connect-standalone.sh ./config/connect-standalone.properties ./config/SF_connect.properties
~略~
[2023-03-08 12:12:11,584] INFO [SF_KAFKA_CONNECTOR] Created loggerHandler for class: 'com.snowflake.kafka.connector.internal.InternalUtils' without a Kafka Connect global instance id. (com.snowflake.kafka.connector.internal.LoggerHandler:64)
[2023-03-08 12:12:11,585] ERROR Failed to create job for ./config/SF_connect.properties (org.apache.kafka.connect.cli.ConnectStandalone:111)
[2023-03-08 12:12:11,598] ERROR Stopping after connector error (org.apache.kafka.connect.cli.ConnectStandalone:121)
java.util.concurrent.ExecutionException: java.lang.NoClassDefFoundError: org/bouncycastle/jcajce/provider/BouncyCastleFipsProvider
at org.apache.kafka.connect.util.ConvertingFutureCallback.result(ConvertingFutureCallback.java:115)
at org.apache.kafka.connect.util.ConvertingFutureCallback.get(ConvertingFutureCallback.java:99)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:118)
Caused by: java.lang.NoClassDefFoundError: org/bouncycastle/jcajce/provider/BouncyCastleFipsProvider
at com.snowflake.kafka.connector.internal.InternalUtils.createProperties(InternalUtils.java:151)
at com.snowflake.kafka.connector.internal.SnowflakeConnectionServiceFactory$SnowflakeConnectionServiceBuilder.setProperties(SnowflakeConnectionServiceFactory.java:58)
at com.snowflake.kafka.connector.SnowflakeSinkConnector.validate(SnowflakeSinkConnector.java:243)
at org.apache.kafka.connect.runtime.AbstractHerder.validateConnectorConfig(AbstractHerder.java:496)
at org.apache.kafka.connect.runtime.AbstractHerder.lambda$validateConnectorConfig$2(AbstractHerder.java:377)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Caused by: java.lang.ClassNotFoundException: org.bouncycastle.jcajce.provider.BouncyCastleFipsProvider
at java.net.URLClassLoader.findClass(URLClassLoader.java:387)
at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
... 10 more
[2023-03-08 12:12:11,601] INFO Kafka Connect stopping (org.apache.kafka.connect.runtime.Connect:66)
~略~
SF_connect.propertiesの内容
name=kafka_connector_stage
connector.class=com.snowflake.kafka.connector.SnowflakeSinkConnector
tasks.max=2
topics=test_topic
snowflake.topic2table.map= test_topic:test_topic_table
buffer.count.records=10000
buffer.flush.time=10
buffer.size.bytes=5000000
snowflake.url.name=*****.ap-northeast-1.aws.snowflakecomputing.com
snowflake.user.name=kafka_connector_user_1
snowflake.private.key=MIIFHzBJBgkqhkiG9w0BBQ0wPDA******hkiG9w0BBQwwDgQIiWRXPv700kICAgg/mA==
snowflake.private.key.passphrase=*****
snowflake.database.name=KAFKA_DB
snowflake.schema.name=KAFKA_SCHEMA
key.converter=com.snowflake.kafka.connector.records.SnowflakeJsonConverter
value.converter=com.snowflake.kafka.connector.records.SnowflakeJsonConverter
実行環境
Kafkaを実行している環境はWindows10 Virtual Machine上に建てたCentOS7.9です。