CDAP deployement

[daxtergithub]

Hi
Thanks for the kubernates operator
We are trying to install CDAP on openshift environment and we used cdap-operator release/1.0 branch
we have been able to install operator on openshift and its working fine ,however weare trying to install cdap using realease/1.0/config/sample/ yaml file but it getting error

2020/04/30 04:28:29 genericreconciler.go:356: *v1alpha1.CDAPMaster/cdap-test/master-1(cmpnt:*controllers.ConfigMapHandler)  } reconciling component
  | E0430 04:28:29.837811       1 genericreconciler.go:115] error reconciling *v1alpha1.CDAPMaster/cdap-test/master-1. [configmaps "cdap-master-1-cconf" is forbidden: cannot set blockOwnerDeletion in this case because cannot find RESTMapping for APIVersion cdap.cdap.io/v1alpha1 Kind *v1alpha1.CDAPMaster: no matches for kind "*v1alpha1.CDAPMaster" in version "cdap.cdap.io/v1alpha1", configmaps "cdap-master-1-hconf" is forbidden: cannot set blockOwnerDeletion in this case because cannot find RESTMapping for APIVersion cdap.cdap.io/v1alpha1 Kind *v1alpha1.CDAPMaster: no matches for kind "*v1alpha1.CDAPMaster" in version "cdap.cdap.io/v1alpha1", configmaps "cdap-master-1-sysappconf" is forbidden: cannot set blockOwnerDeletion in this case because cannot find RESTMapping for APIVersion cdap.cdap.io/v1alpha1 Kind *v1alpha1.CDAPMaster: no matches for kind "*v1alpha1.CDAPMaster" in version "cdap.cdap.io/v1alpha1"]

any idea ?

Thanks

[saneera]

@wyzhang this is happening because of blockOwnerDeletion is there any way to override that using code

[wyzhang]

@daxtergithub This is a known issue with openshift: "OwnerReferences cannot be set on custom resource" openshift/origin#22958. According to openshift/origin#22958 (comment), 4.2 version might work for you.

@saneera Not configurable at the moment. blockOwnerDeletion picks up the default value of true via vendor/sigs.k8s.io/controller-reconciler/pkg/genericreconciler/genericreconciler.go:216

[saneera]

Hi @wyzhang thanks for reply , I have tried to install 4.3 cluster still getting same error
Is there anyway to only use the yaml files to deploy cdap only without operator

[saneera]

@wyzhang
we are trying to use this on openshift 4.2 and still getting the same error and temporarily make this thing to work and pods are spinning up but crashing continuously

Do you have idea what is folowing error means

2020-05-14 05:06:59,910 - ERROR [main:i.c.c.c.r.DaemonMain@68] - Exception raised from main method
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.RuntimeException: Failure response from API service, code=403, body={"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services \"cdap-demo1-dataset-executor\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , \u003cnil\u003e","reason":"Forbidden","details":{"name":"cdap-demo1-dataset-executor","kind":"services"},"code":403}

	at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractService.startAndWait(AbstractService.java:220) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractIdleService.startAndWait(AbstractIdleService.java:106) ~[com.google.guava.guava-13.0.1.jar:na]
	at io.cdap.cdap.master.environment.k8s.AbstractServiceMain.start(AbstractServiceMain.java:199) [io.cdap.cdap.cdap-app-fabric-6.2.0-SNAPSHOT.jar:na]
	at io.cdap.cdap.common.runtime.DaemonMain.doMain(DaemonMain.java:60) ~[io.cdap.cdap.cdap-common-6.2.0-SNAPSHOT.jar:na]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_232]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_232]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_232]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_232]
	at io.cdap.cdap.master.environment.k8s.AbstractServiceMain.main(AbstractServiceMain.java:118) [io.cdap.cdap.cdap-app-fabric-6.2.0-SNAPSHOT.jar:na]
	at io.cdap.cdap.master.environment.k8s.AppFabricServiceMain.main(AppFabricServiceMain.java:79) [io.cdap.cdap.cdap-master-6.2.0-SNAPSHOT.jar:na]
java.lang.RuntimeException: Failure response from API service, code=403, body={"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services \"cdap-demo1-dataset-executor\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , \u003cnil\u003e","reason":"Forbidden","details":{"name":"cdap-demo1-dataset-executor","kind":"services"},"code":403}

	at io.cdap.cdap.k8s.discovery.KubeDiscoveryService.register(KubeDiscoveryService.java:131) ~[na:na]
	at io.cdap.cdap.data2.datafabric.dataset.service.executor.DatasetOpExecutorService.startUp(DatasetOpExecutorService.java:90) ~[na:na]
	at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ~[com.google.guava.guava-13.0.1.jar:na]
	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_232]
Caused by: io.kubernetes.client.ApiException: Forbidden
	at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:882) ~[na:na]
	at io.kubernetes.client.ApiClient.execute(ApiClient.java:798) ~[na:na]
	at io.kubernetes.client.apis.CoreV1Api.createNamespacedServiceWithHttpInfo(CoreV1Api.java:9416) ~[na:na]
	at io.kubernetes.client.apis.CoreV1Api.createNamespacedService(CoreV1Api.java:9398) ~[na:na]
	at io.cdap.cdap.k8s.discovery.KubeDiscoveryService.createV1Service(KubeDiscoveryService.java:270) ~[na:na]
	at io.cdap.cdap.k8s.discovery.KubeDiscoveryService.register(KubeDiscoveryService.java:112) ~[na:na]
	... 3 common frames omitted
2020-05-14 05:06:59,911 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@206] - Stopping all services for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,912 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service MasterTaskExecutorService [NEW] for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,913 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service AppFabricServer [NEW] for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,913 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service EndureService{DatasetService} for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,913 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service TwillRunnerServiceWrapper [NEW] for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,913 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service TwillRunnerServiceWrapper [NEW] for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,913 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service DatasetServiceStore [NEW] for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,914 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service DatasetOpExecutorService{bindAddress=/0:0:0:0:0:0:0:0:35833} for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,917 - WARN  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@213] - Exception raised when stopping service DatasetOpExecutorService{bindAddress=/0:0:0:0:0:0:0:0:35833} for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
com.google.common.util.concurrent.UncheckedExecutionException: java.lang.Exception: Service failed to start.
	at com.google.common.util.concurrent.Futures.wrapAndThrowUnchecked(Futures.java:1015) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.Futures.getUnchecked(Futures.java:1001) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractService.stopAndWait(AbstractService.java:225) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractIdleService.stopAndWait(AbstractIdleService.java:122) ~[com.google.guava.guava-13.0.1.jar:na]
	at io.cdap.cdap.master.environment.k8s.AbstractServiceMain.stop(AbstractServiceMain.java:210) ~[na:na]
	at io.cdap.cdap.common.runtime.DaemonMain$1.run(DaemonMain.java:46) [na:na]
java.lang.Exception: Service failed to start.
	at com.google.common.util.concurrent.AbstractService$1.failed(AbstractService.java:120) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractService$6$1.run(AbstractService.java:444) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.MoreExecutors$SameThreadExecutorService.execute(MoreExecutors.java:262) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractService$ListenerExecutorPair.execute(AbstractService.java:470) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractService$6.run(AbstractService.java:442) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractService.executeListeners(AbstractService.java:369) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractService.notifyFailed(AbstractService.java:313) ~[com.google.guava.guava-13.0.1.jar:na]
	at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:46) ~[com.google.guava.guava-13.0.1.jar:na]
	at java.lang.Thread.run(Thread.java:748) ~[na:1.8.0_232]
Caused by: java.lang.RuntimeException: Failure response from API service, code=403, body={"kind":"Status","apiVersion":"v1","metadata":{},"status":"Failure","message":"services \"cdap-demo1-dataset-executor\" is forbidden: cannot set blockOwnerDeletion if an ownerReference refers to a resource you can't set finalizers on: , \u003cnil\u003e","reason":"Forbidden","details":{"name":"cdap-demo1-dataset-executor","kind":"services"},"code":403}

	at io.cdap.cdap.k8s.discovery.KubeDiscoveryService.register(KubeDiscoveryService.java:131) ~[na:na]
	at io.cdap.cdap.data2.datafabric.dataset.service.executor.DatasetOpExecutorService.startUp(DatasetOpExecutorService.java:90) ~[na:na]
	at com.google.common.util.concurrent.AbstractIdleService$1$1.run(AbstractIdleService.java:43) ~[com.google.guava.guava-13.0.1.jar:na]
	... 1 common frames omitted
Caused by: io.kubernetes.client.ApiException: Forbidden
	at io.kubernetes.client.ApiClient.handleResponse(ApiClient.java:882) ~[na:na]
	at io.kubernetes.client.ApiClient.execute(ApiClient.java:798) ~[na:na]
	at io.kubernetes.client.apis.CoreV1Api.createNamespacedServiceWithHttpInfo(CoreV1Api.java:9416) ~[na:na]
	at io.kubernetes.client.apis.CoreV1Api.createNamespacedService(CoreV1Api.java:9398) ~[na:na]
	at io.cdap.cdap.k8s.discovery.KubeDiscoveryService.createV1Service(KubeDiscoveryService.java:270) ~[na:na]
	at io.cdap.cdap.k8s.discovery.KubeDiscoveryService.register(KubeDiscoveryService.java:112) ~[na:na]
	... 3 common frames omitted
2020-05-14 05:06:59,918 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service DefaultSecureStoreService [RUNNING] for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,919 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service OperationalStatsService [RUNNING] for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:06:59,919 - WARN  [OperationalStatsService:i.c.c.o.OperationalStatsService@144] - Failed to collect stats for service CDAP of type lastHourLoad due to Unable to discover metrics service endpoint
2020-05-14 05:06:59,920 - INFO  [OperationalStatsService:i.c.c.o.OperationalStatsService@183] - Successfully shutdown operational stats service.
2020-05-14 05:06:59,920 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@208] - Stopping service MessagingMetricsCollectionService [RUNNING] for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain
2020-05-14 05:07:06,934 - ERROR [MessagingMetricsCollectionService:i.c.c.m.c.AggregatedMetricsCollectionService@135] - Failed in publishing metrics for timestamp 1589432819.
java.io.IOException: Failed to publish metrics to TMS and exceeded retry limit.
	at io.cdap.cdap.metrics.collect.MessagingMetricsCollectionService$TopicPayload.publish(MessagingMetricsCollectionService.java:164) ~[na:na]
	at io.cdap.cdap.metrics.collect.MessagingMetricsCollectionService.publishMetric(MessagingMetricsCollectionService.java:107) ~[na:na]
	at io.cdap.cdap.metrics.collect.MessagingMetricsCollectionService.publish(MessagingMetricsCollectionService.java:102) ~[na:na]
	at io.cdap.cdap.metrics.collect.AggregatedMetricsCollectionService.publishMetrics(AggregatedMetricsCollectionService.java:133) [na:na]
	at io.cdap.cdap.metrics.collect.AggregatedMetricsCollectionService.shutDown(AggregatedMetricsCollectionService.java:166) [na:na]
	at com.google.common.util.concurrent.AbstractExecutionThreadService$1$1.run(AbstractExecutionThreadService.java:65) [com.google.guava.guava-13.0.1.jar:na]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_232]
Caused by: io.cdap.cdap.common.ServiceUnavailableException: Service 'messaging.service' is not available. Please wait until it is up and running.
	at io.cdap.cdap.common.internal.remote.RemoteClient.resolve(RemoteClient.java:112) ~[na:na]
	at io.cdap.cdap.common.internal.remote.RemoteClient.requestBuilder(RemoteClient.java:73) ~[na:na]
	at io.cdap.cdap.messaging.client.ClientMessagingService.performWriteRequest(ClientMessagingService.java:250) ~[na:na]
	at io.cdap.cdap.messaging.client.ClientMessagingService.publish(ClientMessagingService.java:186) ~[na:na]
	at io.cdap.cdap.metrics.collect.MessagingMetricsCollectionService$TopicPayload.publish(MessagingMetricsCollectionService.java:154) ~[na:na]
	... 6 common frames omitted
2020-05-14 05:07:08,561 - INFO  [Thread-4:i.c.c.m.e.k.AbstractServiceMain@225] - All services for io.cdap.cdap.master.environment.k8s.AppFabricServiceMain stopped
2020-05-14 05:07:08,562 - INFO  [Thread-4:i.c.c.m.e.k.KubeMasterEnvironment@160] - Kubernetes environment destroyed