Skip to content

Latest commit

 

History

18,916 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

SEMOSS

An open source platform for building AI applications, connecting data, and running analytics.

SEMOSS (Semantic Open Source Software) brings models, databases, vector stores, storage providers, and custom functions into a common framework. Use it to build AI assistants and applications, work with enterprise data, and create repeatable analytical workflows through a web interface, APIs, or code.

This repository contains the core SEMOSS runtime: the Java execution engine, Pixel language and reactors, engine integrations, Python and R components, and container build definitions. Monolith exposes the runtime through a web application and APIs, and semoss-ui provides the user interface.

Documentation · Docker quick start · Build from source · Complete deployment examples · Issues

Contents

Capabilities

  • AI and model integration. Connect language and embedding models through a shared engine interface. Combine model calls with data retrieval, functions, and application workflows.
  • Data access and analytics. Query connected databases, combine data from multiple sources in frames, transform results, and use them in applications, notebooks, and automated workflows.
  • Vector search and retrieval. Ingest documents and connect vector databases for semantic search and retrieval augmented generation.
  • Applications and automation. Build applications, agents, and reusable workflows using Pixel, Java reactors, Python, R, and custom functions or APIs.
  • Shared access controls. Manage access to engines, projects, and insights, with platform services for authentication, permissions, model usage logging, and auditing.
  • Flexible deployment. Run locally with Docker Compose or deploy multiple application nodes with shared databases, object storage, and Redis or ZooKeeper coordination.

Available integrations and runtime features depend on the engines and services configured in your installation.

Architecture and related repositories

flowchart LR
    UI["semoss-ui"] --> Web["Monolith: web APIs and sessions"]
    Clients["API clients"] --> Web
    Web --> Core["Semoss: core runtime"]
    Core --> Engines["Models, databases, vectors, storage, functions"]
    Core --> Runtimes["Python and R runtimes"]
    Core --> State["System databases and shared assets"]
Loading

Monolith packages the core runtime as a Java dependency in its WAR. These are layers of the application; the diagram does not require a separate service for each box.

Repository Responsibility Start here
Semoss — this repository Core execution, connectors, data processing, and runtime assets Backend documentation
Monolith Java web application, HTTP APIs, authentication integration, sessions, and WebSockets README and local development
semoss-ui Frontend applications and shared UI libraries Frontend setup
SEMOSS-deployment Complete Kubernetes deployment examples and infrastructure configuration Deployment guide

For a ready-to-run local platform, use the Docker examples below. To run backend code changes, use Monolith's local Docker examples.

Quick start with Docker

The quickest way to try SEMOSS is the Docker Compose examples. They use a published SEMOSS image containing the web application and UI, so no local Java or frontend build is required.

Prerequisites

  • Git.
  • Docker with a running daemon and Docker Compose v2 (docker compose).
  • Access to pull container images, including the SEMOSS image from Quay.io.
  • Available host ports 9090 for SEMOSS and 5432 for PostgreSQL.

Start a single node

git clone https://github.com/SEMOSS/Semoss.git
cd Semoss/docker-compose-examples

# Create the shared network if it does not already exist.
docker network inspect semoss-net >/dev/null 2>&1 || docker network create semoss-net

docker compose -f semoss-with-postgres.yml up -d
docker compose -f semoss-with-postgres.yml logs -f semoss

Wait for application startup, then open http://localhost:9090/#/. The example enables native registration for local use; follow the application's account setup and sign-in flow. PostgreSQL credentials in the Compose file are database credentials, not a SEMOSS user account.

This stack starts SEMOSS and PostgreSQL, initializes the system databases using init.sql, and persists application data in named Docker volumes. Python is enabled and R is disabled in these examples.

Check the stack or stop it from the same directory:

docker compose -f semoss-with-postgres.yml ps
docker compose -f semoss-with-postgres.yml down

down retains named volumes. Adding -v deletes those volumes and their data.

Choose an example

Compose file Topology Application URLs
semoss-with-postgres.yml One SEMOSS node and PostgreSQL http://localhost:9090/#/
semoss-with-postgres-minio.yml One node, PostgreSQL, and MinIO shared asset storage http://localhost:9090/#/
semoss-with-postgres-minio-redis.yml Two nodes, PostgreSQL, MinIO, and Redis synchronization Ports 9090 and 9091
semoss-with-postgres-minio-zk.yml Two nodes, PostgreSQL, MinIO, and ZooKeeper synchronization Ports 9090 and 9091

Use the selected filename with docker compose -f. Run one platform example at a time: the stacks share fixed container names and host ports, including with Monolith's local examples. Stop the current stack before switching. Cluster variants use service names semoss1 and semoss2 for logs.

The Compose guide documents supporting service ports, storage behavior, and configuration. The examples use development images and local credentials; configure credentials, authentication, and network exposure for your deployment before making it accessible to others.

Add supporting engines

The engine examples provide optional services and connection instructions for vector databases, ClickHouse, object storage, SFTP, and mail functions. Start only the services you need, then configure the corresponding engine in SEMOSS. These examples share the semoss-net Docker network with the platform examples.

Complete deployments

For Kubernetes and infrastructure deployment, or semoss-artifacts property configuration, see SEMOSS-deployment.

Core concepts

Concept Purpose Documentation
Pixel Language for expressing data operations, application actions, and execution workflows Pixel language
Reactors Java implementations of operations invoked by Pixel Reactor framework
Engines Common abstraction for databases, models, vector stores, storage, and functions Engine abstraction
Frames and query structures Represent data and describe queries and transformations DataFrames and QueryStructs
Insights Hold execution context, results, and analytical state The Insight object
Projects Organize application assets and reusable work Project engines

A typical request travels from semoss-ui or an API client through Monolith to the core runtime. Pixel operations invoke reactors, which use engines and frames to perform the work. Results return through Monolith to the caller. See the backend architecture and Monolith integration.

Building from source

Requirements

Use JDK 21 and Apache Maven 3.9.x; the Java compiler targets 21 and CI uses Maven 3.9.9. Git and access to the Maven repositories declared in pom.xml are also required. Check the active toolchain with java -version and mvn -version.

From the repository root:

mvn clean install

This compiles the core, runs the default test selection, creates artifacts under target/, and installs them into your local Maven repository. For a build that skips test execution:

mvn clean install -DskipTests

The default dev profile is intended for local development. The deploy profile adds release packaging and signing steps; it is not needed for a local build. The development build also configures the repository's Git hooks.

Run your changes through Monolith

Keep the two repositories next to each other:

workspace/
├── Semoss/
└── Monolith/

Build Semoss before Monolith. Their ci.version values must match because Monolith depends on org.semoss:semoss with the shaded-dependencies classifier. A successful mvn install here makes that artifact available to the local Monolith build.

Follow Monolith's Docker quick start to build both repositories and run the resulting local-monolith image. The published-image examples in this repository do not include unbuilt local source changes. Frontend development is documented in semoss-ui.

Running tests

Run these commands from the repository root:

# Run the default unit test selection.
mvn test

# Run one test class.
mvn -Dtest=YearReactorUnitTests test

# Run verification, including the configured JaCoCo report step.
mvn verify

The default Maven profile selects **/*UnitTests.java from test/. This is not a promise that every integration test or external service is exercised. Tests for specific integrations may require additional services and configuration. Surefire results are written to target/surefire-reports/, and JaCoCo reports to target/site/jacoco/ when generated.

Configuration

The running application uses a SEMOSS home directory for configuration, engine definitions, projects, and runtime assets. Container startup translates supported environment variables into the application's configuration; use the checked-in Compose files as concrete examples.

Configuration area Where to look
Core runtime, feature flags, and engine settings RDF_Map.prop and configuration guide
Login and identity provider settings social.properties and authentication and authorization
Application logging log4j2.xml
System databases Internal databases and Compose initialization SQL
Local Docker ports, networks, volumes, and service connections Local Docker configuration and Compose examples
Container startup and semoss-artifacts property configuration SEMOSS-deployment
Shared storage and cluster operation Cloud and cluster documentation and deployment examples

In the examples, CUSTOM_* variables configure the system databases; SETSOCIAL, ENABLE_NATIVE, and REDIRECT configure local login behavior; NETTY_PYTHON and NATIVE_PY_SERVER enable Python integration. MinIO variants add shared storage configuration, and the two-node variants add a coordination backend. Preserve the database names and connection settings consistently with the initialization SQL.

Repository layout

Path Contents
src/prerna/reactor/ Pixel operations and application logic
src/prerna/sablecc2/ Pixel parsing and execution support
src/prerna/engine/ Engine interfaces and integrations
src/prerna/ds/ and src/prerna/query/ Frames, query structures, and query interpretation
src/prerna/auth/ Identity and permission services
src/prerna/cluster/ Shared storage and cluster coordination
py/, R/, and js/ Language runtime components and supporting scripts
test/ Java tests and test resources
docker/ Runtime and application image build definitions
docker-compose-examples/ Local platform stacks and supporting engine examples
docs/ Architecture, concepts, integrations, and development guides
hooks/ and dev-scripts/ Contributor tooling

Documentation

Start with the documentation index, then choose a path:

Troubleshooting

Symptom What to check
Compose reports that semoss-net does not exist Create the external network using the quick-start command.
A container name or port is already in use Stop the other SEMOSS or Monolith example; check for a local PostgreSQL service on port 5432.
The UI is unavailable during first startup Check docker compose -f semoss-with-postgres.yml ps and logs for both semoss and db; image downloads and database initialization can take time.
Login redirects to the wrong address Match REDIRECT to the browser-facing URL and check authentication and cookie settings.
PostgreSQL settings changed but the old configuration remains PostgreSQL initialization scripts run only on a fresh data directory. Update the existing database explicitly or recreate disposable development data.
Java compilation reports an unsupported release Confirm mvn -version uses JDK 21.
Local changes do not appear in the running application Build the core and Monolith, rebuild local-monolith, and recreate its container using Monolith's development workflow.

Contributing and support

Bug reports, documentation improvements, tests, and new integrations are welcome. Use Semoss issues for core runtime problems, Monolith issues for web/API problems, and semoss-ui issues for frontend problems.

For a change, keep the scope focused, include relevant tests and documentation, and describe the behavior and validation in your pull request. Follow the commit message conventions. When reporting a bug, include the revision or image tag, deployment topology, reproduction steps, and relevant logs with credentials and private data removed.

License

See LICENSE for the Apache License 2.0 terms. Retain applicable source-file and third-party dependency notices when redistributing the software.

About

No description, website, or topics provided.

Resources

Contributing

Stars

55 stars

Watchers

8 watching

Forks

Releases

Packages

Used by

Contributors

Languages