[patch] MAXMIS-5210: Add Kafka requirement for Manage Civil Infrastructure component

[bhavsar909]

Description

Add Kafka configuration requirement for Maximo Manage Civil Infrastructure component, extending the existing IoT Kafka validation logic to also support Civil Infrastructure installations.

Changes Made

Core Logic Updates
python/src/mas/cli/install/app.py: Enhanced Kafka validation logic
Added detection for Civil Infrastructure component (civil= in mas_appws_components)
Extended Kafka requirement check to trigger for both IoT and Civil Infrastructure
Improved error messages to provide context-specific guidance:
Combined message when both IoT and Civil are enabled
IoT-specific message when only IoT requires Kafka
Civil-specific message when only Civil Infrastructure requires Kafka

Configuration Settings
python/src/mas/cli/install/settings/kafkaSettings.py: Updated Kafka configuration prompts
Added installManage attribute to type hints
Modified configKafka() method to detect Civil Infrastructure component
Dynamic description generation based on which components require Kafka
Updated user-facing messages to reflect both IoT and Civil Infrastructure requirements

Pipeline Updates
tekton/src/pipelines/taskdefs/dependencies/kafka.yml.j2: Updated pipeline documentation
Clarified comment to reflect that Kafka is required for IoT OR Manage with Civil Infrastructure

Test Results

Tested using cli from cmd as follows:

`
[ibmmas/cli:20.2.0-pre.cikar]mascli$ mas install --dev-mode

IBM Maximo Application Suite Admin CLI v20.2.0-pre.cikar
Powered by https://github.com/ibm-mas/ansible-devops/ and https://tekton.dev/

1. Set Target OpenShift Cluster
   Already connected to OCP Cluster:
   https://console-openshift-console.apps.civilinf-raj-qb1.cp.fyre.ibm.com

Proceed with this cluster? [y/n] y

2. Choose Install Mode
   There are two flavours of the interactive install to choose from: Simplified and Advanced. The simplified option will present fewer dialogs, but you lose the ability to configure the following aspects of the installation:

• Configure installation namespaces
• Provide pod templates
• Configure Single Sign-On (SSO) settings - Configure whether to trust well-known certificate authorities by default (defaults to enabled)
• Configure whether the Guided Tour feature is enabled (defaults to enabled)
• Configure whether special characters are allowed in usernames and userids (defaults to disabled)
• Configure a custom domain, DNS integrations, routing mode and manual certificates
• Customize Maximo Manage database settings (schema, tablespace, indexspace)
• Customize Maximo Manage server bundle configuration (defaults to "all" configuration)
• Enable optional Maximo Manage integration Cognos Analytics and Watson Studio Local
• Enable optional Real Estate and Facilities configurations
• Customize Db2 node affinity and tolerations, memory, cpu, and storage settings (when using the IBM Db2 Universal Operator)
• Choose alternative Apache Kafka providers (default to Strimzi)
• Customize Grafana storage settings
  Show advanced installation options? [y/n] n

3. IBM Maximo Operator Catalog Selection
   Select catalog source v9-civil-amd64
   Select channel 9.2.x-dev

4. Configure Storage Class Usage
   Maximo Application Suite and it's dependencies require storage classes that support ReadWriteOnce (RWO) and ReadWriteMany (RWX) access modes:

• ReadWriteOnce volumes can be mounted as read-write by multiple pods on a single node.
• ReadWriteMany volumes can be mounted as read-write by multiple pods across many nodes.

Storage provider auto-detected: NFS Client

• Storage class (ReadWriteOnce): nfs-client
• Storage class (ReadWriteMany): nfs-client
  Use the auto-detected storage classes? [y/n] y

5. Configure AppPoint Licensing
   By default the MAS instance will be configured to use a cluster-shared License, this provides a shared pool of AppPoints available to all MAS instances on the cluster.

License file /home/mas-config/entitlement.lic
SLS channel 3.x-dev
Contact e-mail address xxx@xxx.com
Contact first name xxx
Contact last name xxxx

6. Configure IBM Container Registry
   IBM entitlement key ********************************************************************************************************************************************************************************************************************************

---

---

---

Artifactory username civil.maximo@de.ibm.com
Artifactory token **********************************************************************************************************************************************************************************************************************************

---

---

---

7. Configure MAS Instance
   Instance ID restrictions:

• Must be 3-12 characters long
• Must only use lowercase letters, numbers, and hypen (-) symbol
• Must start with a lowercase letter
• Must end with a lowercase letter or a number
  Instance ID civil

Workspace ID restrictions:

• Must be 3-12 characters long
• Must only use lowercase letters and numbers
• Must start with a lowercase letter
  Workspace ID civil

Workspace display name restrictions:

• Must be 3-300 characters long
  Workspace name main

8. Configure Operational Mode
   Maximo Application Suite can be installed in a non-production mode for internal development and testing, this setting cannot be changed after installation:

• All applications, add-ons, and solutions have 0 (zero) installation AppPoints in non-production installations.
• These specifications are also visible in the metrics that are shared with IBM and in the product UI.

1. Production
2. Non-Production
   Operational Mode 1

9. Application Selection
   Install IoT? [y/n] n
   Install Monitor? [y/n] n
   Install Manage? [y/n] y
   Custom channel for manage9.2.x-dev-civil-rbac
   Install Assist? [y/n] n
   Install Optimizer? [y/n] n
   Install Visual Inspection? [y/n] y
   Custom channel for visualinspection9.2.x-dev
   Install Real Estate and Facilities? [y/n] n
   Install AI Service? [y/n] n

10. Configure Maximo Manage
    Customize your Manage installation, refer to the product documentation for more information

10.1) Maximo Manage Components
The default configuration will install Manage with Health enabled, alternatively choose exactly what industry solutions and add-ons will be configured
Select components to enable? [y/n] y

• Asset Configuration Manager? [y/n] n
• Aviation? [y/n] n
• Civil Infrastructure? [y/n] y
• Envizi? [y/n] n
• Health? [y/n] n
• Health, Safety and Environment? [y/n] n
• Maximo IT? [y/n] n
• Nuclear? [y/n] n
• Oil & Gas? [y/n] n
• Connector for Oracle Applications? [y/n] n
• Connector for SAP Application? [y/n] n
• Service Provider? [y/n] n
• Spatial? [y/n] n
• Strategize? [y/n] n
• Transportation? [y/n] n
• Tririga? [y/n] n
• Utilities? [y/n] n
• Workday Applications? [y/n] n
• AIP? [y/n] n
• Vegetation Management? [y/n] n

10.2) Maximo Manage Settings - Customization
Provide a customization archive to be used in the Manage build process
Include customization archive? [y/n] n

10.3) Maximo Manage Settings - Other

11. Configure MongoDb
    The installer can setup mongoce in your OpenShift cluster (available only for amd64) or you may choose to configure MAS to use an existing mongodb
    Create MongoDb cluster using MongoDb Community Edition Operator? [y/n] y

12. Configure Databases
    The installer can setup one or more IBM Db2 instances in your OpenShift cluster for the use of applications that require a JDBC datasource (IoT, Manage, Monitor, & Predict, Real Estate and Facilities) or you may choose to configure MAS to use an existing database

12.1) Database Configuration for Maximo Manage
Maximo Manage can be configured to share the system Db2 instance or use it's own dedicated database:

• Use of a shared instance has a significant footprint reduction but is only recommended for development/test/demo installs
• In most production systems you will want to use a dedicated database
• IBM Db2, Oracle Database, & Microsoft SQL Server are all supported database options
  Create Manage dedicated Db2 instance using the IBM Db2 Universal Operator? [y/n] y

13. Configure Kafka
    Maximo Manage Civil Infrastructure requires a shared system-scope Kafka instance
    Supported Kafka providers: Strimzi, Red Hat AMQ Streams, IBM Cloud Event Streams and AWS MSK
    You may also choose to configure MAS to use an existing Kafka instance by providing a pre-existing configuration file
    Create system Kafka instance using one of the supported providers? [y/n] y

Strimzi: Cluster Version
The version of the Strimzi operator available on your cluster will determine the supported versions of Kafka that can be deployed.

• If you are using the latest available operator catalog then the default version below can be accepted
• If you are using older operator catalogs (e.g. in a disconnected install) you should confirm the supported versions in your OperatorHub
  Kafka version 3.9.0
  Install Grafana? [y/n] y

14. Additional Configuration
    Additional resource definitions can be applied to the OpenShift Cluster during the MAS configuration step
    The primary purpose of this is to apply configuration for Maximo Application Suite itself, but you can use this to deploy ANY additional resource into your cluster
    Use additional configurations? [y/n] n

15. Non-Interactive Install Command
    Save and re-use the following script to re-run this install without needing to answer the interactive prompts again

export IBM_ENTITLEMENT_KEY=x
export ARTIFACTORY_USERNAME=x
export ARTIFACTORY_TOKEN=x
mas install --mas-catalog-version v9-civil-amd64 --ibm-entitlement-key $IBM_ENTITLEMENT_KEY
--mas-channel 9.2.x-dev --mas-instance-id civil --mas-workspace-id civil --mas-workspace-name "main"
--servicemesh "false"
--storage-class-rwo "nfs-client" --storage-class-rwx "nfs-client"
--storage-pipeline "nfs-client" --storage-accessmode "ReadWriteMany"
--sls-channel "3.x-dev" --license-file "/home/mas-config/entitlement.lic"
--contact-email "xxx@xxx.com" --contact-firstname "xxx" --contact-lastname "xxxx"
--mongodb-namespace "mongoce"
--manage-channel "9.2.x-dev-civil-rbac"
--visualinspection-channel "9.2.x-dev"
--manage-jdbc "workspace-application"
--manage-components "base=latest,civil=latest"
--manage-server-bundle-size "dev"
--db2-manage
--db2-channel "v110509.0"
--db2-namespace "db2u"
--db2-type "db2wh"
--db2-cpu-requests "4000m"
--db2-cpu-limits "6000m"
--db2-memory-requests "8Gi"
--db2-memory-limits "12Gi"
--db2-backup-storage "50Gi"
--db2-data-storage "50Gi"
--db2-logs-storage "10Gi"
--db2-meta-storage "10Gi"
--db2-temp-storage "10Gi"
--kafka-provider "strimzi"
--kafka-version "3.9.0"
--artifactory-username $ARTIFACTORY_USERNAME --artifactory-token $ARTIFACTORY_TOKEN
--dev-mode
--accept-license --no-confirm

16. Review Settings
    Connected to:

• https://console-openshift-console.apps.civilinf-raj-qb1.cp.fyre.ibm.com

16.1) Pipeline Configuration
Service Account ......................... Default
Image Pull Policy ....................... Default
Skip Pre-Install Healthcheck ............ No

16.2) OpenShift Container Platform
Worker Node Architecture ................ amd64
Storage Class Provider .................. nfs
ReadWriteOnce Storage Class ............. nfs-client
ReadWriteMany Storage Class ............. nfs-client
Certificate Manager ..................... redhat
Cluster Ingress Certificate Secret ...... Default
Single Node OpenShift ................... No

16.3) IBM Data Reporter Operator (DRO) Configuration
Contact e-mail .......................... xxx@xxx.com
First name .............................. xxx
Last name ............................... xxxx
Install Namespace ....................... Default

16.4) IBM Suite License Service
Namespace ............................... Default
Subscription Channel .................... 3.x-dev
IBM Open Registry ....................... docker-na-public.artifactory.swg-devops.com/wiotp-docker-local/cpopen
License File ............................ /home/mas-config/entitlement.lic

16.5) IBM Maximo Application Suite
Instance ID ............................. civil
Workspace ID ............................ civil
Workspace Name .......................... main

Operational Mode ........................ Production
Install Mode ............................ Connected Install

Network Routing Mode .................... Default

Use Service Mesh ........................ false

Configure Suite to run in IPV6 .......... Default

Manual Certificates ..................... Not Configured

Enable Guided Tour ...................... Default

Catalog Version ......................... v9-civil-amd64
Subscription Channel .................... 9.2.x-dev

IBM Entitled Registry ................... docker-na-public.artifactory.swg-devops.com/wiotp-docker-local
IBM Open Registry ....................... docker-na-public.artifactory.swg-devops.com/wiotp-docker-local/cpopen

Enable feature adoption metrics ......... Default

Enable deployment progression metrics ... Default

Enable usability metrics ................ Default

Trust Default Cert Authorities .......... true

Additional Config ....................... Not Configured
Pod Templates ........................... Not Configured

16.6) IBM Maximo Application Suite Applications
IoT ..................................... Do Not Install
Monitor ................................. Do Not Install
Manage .................................. 9.2.x-dev-civil-rbac

• Components
  • ACM ................................. Disabled
  • Aviation ............................ Disabled
  • Civil Infrastructure ................ Enabled
  • Envizi .............................. Disabled
  • Health .............................. Disabled
  • HSE ................................. Disabled
  • Maximo IT ........................... Disabled
  • Nuclear ............................. Disabled
  • Oil & Gas ........................... Disabled
  • Connector for Oracle ................ Disabled
  • Connector for SAP ................... Disabled
  • Service Provider .................... Disabled
  • Spatial ............................. Disabled
  • Strategize .......................... Disabled
  • Transportation ...................... Disabled
  • Tririga ............................. Disabled
  • Utilities ........................... Disabled
  • Workday Applications ................ Disabled
  • AIP ................................. Disabled
  • Vegetation Management ............... Disabled
• Upgrade Type .......................... Default
• Server bundle size .................... dev
• Enable JMS queues ..................... Default
• Server Timezone ....................... Default
• Base Language ......................... Default
• Additional Languages .................. Default
• Database Settings
  • Schema .............................. Default
  • Tablespace .......................... Default
  • Indexspace .......................... Default
    Loc Srv Esri (arcgis) ................... Do Not Install
    Predict ................................. Do Not Install
    Optimizer ............................... Do Not Install
    Assist .................................. Do Not Install
    Visual Inspection ....................... 9.2.x-dev
• Storage Class ........................ nfs-client
  Facilities .............................. Do Not Install

16.7) MongoDb
Type .................................... MongoCE Operator
Install Namespace ....................... mongoce

16.8) IBM Db2 Univeral Operator Configuration
System Instance ......................... Do Not Install
Dedicated Manage Instance ............... Install

• Type ................................. db2wh
• Timezone ............................. Default

Install Namespace ....................... db2u
Subscription Channel .................... v110509.0

CPU Request ............................. 4000m
CPU Limit ............................... 6000m
Memory Request .......................... 8Gi
Memory Limit ........................... 12Gi

Meta Storage ............................ 10Gi
Data Storage ............................ 50Gi
Backup Storage .......................... 50Gi
Temp Storage ............................ 10Gi
Transaction Logs Storage ................ 10Gi

Node Affinity ........................... None
Node Tolerations ........................ None

16.9) Cloud Object Storage
Type .................................... None

16.10) Kafka
Provider ................................ strimzi
Version ................................. 3.9.0
Install Namespace ....................... Default

16.11) Grafana
Install Grafana ......................... Install
Grafana namespace ....................... Default
Grafana storage size .................... Default

16.12) Slack Integration
Slack Channel ........................... Not Configured

Please carefully review your choices above, correcting mistakes now is much easier than after the install has begun
Proceed with these settings? [y/n] y

17. Launch Install
    ✅️ OpenShift Pipelines Operator is installed and ready to use
    ✅️ Namespace is ready (mas-civil-pipelines)
    ✅️ MAS CLI image deployment test completed
    ✅️ Latest Tekton definitions are installed (v20.2.0-pre.cikar)
    ✅️ PipelineRun for civil install submitted

View progress:
https://console-openshift-console.apps.civilinf-raj-qb1.cp.fyre.ibm.com/k8s/ns/mas-civil-pipelines/tekton.dev~v1beta1~PipelineRun/civil-install-260428-0610

[ibmmas/cli:20.2.0-pre.cikar]mascli$
`

Pipeline run from cluster:

[whitfiea]

Is this a hard requirement for civil to have Kafka? If I didn't want to use the kafka imageprocessor then this would still say Kafka is required

[bhavsar909]

Is this a hard requirement for civil to have Kafka? If I didn't want to use the kafka imageprocessor then this would still say Kafka is required

yes, It is hard requirement for 9.2