This section explains, in detail, the design of Prometheus.

Roles and Entitlements

Introduction

The concept of roles and entitlements are central to the way in which prometheus works. DICE has successfully used roles and entitlements since its inception, although entitlements in the DICE system are known as "capabilities" (we have moved away from the use of the word capabilities, as the term is heavily overloaded in the account management world).

Simply put, entities possess roles, which tell us something about the entity itself - e.g. a person entity could possess the role of 'staff'. These roles then contain lists of entitlements (or other roles) - e.g the staff role could (and does) possess the entitlement 'login/staff/remote', which grants permission to log in to staff machines remotely. Entitlements can also be directly applied to an entity.

Management

Roles and entitlements are managed using the rfe system tool, e.g. to edit the 'sysman' role use rfe roles/sysman. Each roles file has the following format:

  • Lines beginning with '#' are treated as comments.
  • Comments which begin with the string doc: are treated as documentation and stored in Prometheus.
  • Whitespace is ignored.
  • Lines beginning with '@' represent the inclusion of another role, e.g. an entry of @staff would include the contents of the roles/staff role.
  • All other lines are treated as entitlements held by that role. The syntax of entitlements is free-form, but we use '/' to separate entitlements into logical groupings (e.g. web/blog/create, printing/colour/print).
  • An entitlement preceded by a '-' is a negated entitlement, so will remove this entitlement from anyone possessing it.
  • An entitlement beginning group/ confers membership of that unix group (e.g. group/sysman).
  • An entitlement preceded by a '*' is a fixed entitlement
  • An entitlement preceded by a '!' is a no-grace entitlement
  • An entitlement with no prefix is a preserved entitlement

For a full description of the latter three items and how they are used in lifecycle processing, see PrometheusLifecycle.

Where the same entitlement is specified multiple times, with different modifier prefixes, the following precedence is enforced:

negated (-) > nograce (!) > fixed (*) > preserved

The roles rfe maps have associated help text, which can be viewed with rfe -xH roles/<rolename>

The existence of a roles rfe file (and hence a corresponding object in ou=Roles) will lead to anyone possessing that role also possessing the entitlement role/<rolename>. This can be overridden by adding -role/<rolename> to the relevant roles rfe file. Similarly, adding !role/<rolename> will mean ownership of the canonical role entitlement will not extend into a user's grace period.

Our roles structure is somewhat convoluted, due in large part to its organic growth over several years. This was partially addressed by the Review roles mechanism development project. A RolesSupremo has been appointed.

It was agreed during an August 2011 meeting that all units should be responsible for the roles/entitlements provision for the services which they provide.

Primary/secondary (upstream/additional) roles/entitlements

Our account management system has the concept of primary and secondary roles and entitlements. This matter is somewhat confused in prometheus itself by the names 'upstream' and 'additional' roles and entitlements. This is explained in more detail later on. Simply put, primary roles are those provided directly from the Informatics database. Any roles/entitlements which do not originate directly from the database are deemed to be secondary/additional.

Roles/Entitlements in Prometheus

Several entitlements have special meaning to prometheus. This section describes them....

  • Any entitlement of the form group/<name> confers membership of the unix group '<name>'. The group must exist in prometheus's GroupStore and have a gid (the GroupStore is populated from the groups rfe map).
  • prometheus/localIdentity - this entitles a user to an identity on local systems (in reality this corresponds to an entry in the KDC) and is necessary for additional account objects (e.g. AFS). Note that during the early stages of prometheus, this entitlement triggered all account provisioning.
  • prometheus/afsHomeDirectory - entitles a user to an AFS home directory
  • prometheus/afsUser - entitles a user to an AFS PTS entry
  • prometheus/ldapPerson - entitles a user to a system ldap (rfc2307) entry for under ou=People
  • prometheus/password/webreset - allows a user to reset their password using the password portal. This entitlement is usually allocated and removed manually, either in roles, or directly on user entitities. e.g. it has been used to allow all new first year students to reset their password during the first four weeks of term.
  • kerberos/policy/<name> - sets KDC password policy to '<name>'
  • prometheus/grace:X - sets the lifecycle grace period for a user to X days
  • prometheus/suspension:X - sets the lifecycle suspension period for a user to X days
  • afs/loc/(student|staff) - used by the afsvol-master conduit to allocate a location for the home directory volume
  • afs/undercommit - used by the afsvol-master conduit to determine if a volume group can be overcommitted

Account-Granting Roles

Posession of any of the following roles qualifies a user for a full DICE account on our systems - this is achieved via the inclusion of dice-account-holder role, which corresponds to entries in the KDC, AFS pts, AFS vldb and system LDAP:

  • cohort-pgr
  • cohort-pgt
  • cohort-pt
  • cohort-ug
  • cohort-vug
  • new-staff
  • staff
  • new-tempvisitor
  • tempvisitor
  • new-visitingstudent
  • visitingstudent

The cohort-* roles should include all students. Prior to their introduction, the numerous (and constantly changing) degree-* and new-degree* roles were used to confer account-granting entitlements.

The dice-account-holder role looks like this:

[bolt]toby: rfe -g roles/dice-account-holder
# role to provide all entitlements for a full dice account

# entitlement to trigger creation of a prometheus identity object (and
# hence an entry in the KDC)
*prometheus/localIdentity

# entitlement to trigger creation of an AFS home dir
*prometheus/afsHomeDirectory

# entitlement to trigger creation of AFS pts entry
*prometheus/afsUser

# entitlement to trigger creation of system ldap (rfc 2307) entry for
# user under ou=People
*prometheus/ldapPerson
[bolt]toby: 

Entitlements and Account Provisioning

The following rules summarise the prometheus approach to account provisioning, based on entitlements:

  • An account object represents any kind of computing account which requires information not found in the entity/identity objects. Any identity may have 0, 1 or many accounts. An identity object alone is required for creation of KDC principals.
  • Population of downstream stores based on the account object (afspts, afsvol, systemldap) will depend on possession, by the account object, of a suitable entitlement (e.g. prometheus/afsHomeDirectory).
  • The base entitlement for DICE accounts is the 'prometheus/localIdentity' entitlement set on the identity object - this entitles a user to a principal in the KDC.
  • When a user loses the 'prometheus/localIdentity' entitlement, they are deemed to have lost the right to a DICE account.

Data Structure and Data Flows

This diagram shows how data flows in and out of the prometheus system.

Central to the operation of prometheus are "datastores" and "conduits".

Datastores are prometheus representations of a source of data, e.g. the following all have corresponding datastores: entities in prometheus, principals in our KDC, person information in the Informatics database, lookups to the IS UID server. The datastores ensure that the data can be accessed and modified (if necessary - some stores are read-only) using a consistent interface.

Conduits are used to link datastores together and it is through conduits that data flows are established. As an example, the InfDB conduit links together the InfDB datastore and the prometheus entity datastore. It is a one-way conduit which takes relevant information from the Informatics database, uses this to construct other bits of information and modifies entities in prometheus accordingly.

Olympus Datastores

This section describes the structure of the data held within Prometheus. We refer to the LDAP prometheus data stores collectively as Olympus, to differentiate them from Prometheus as a wider system.

The key datastores within Olympus are as follows. They all live at the LDAP search base "o=Prometheus,dc=inf,dc=ed,dc=ac,dc=uk".

  • EntityTreeStore (ou=Entities). See this diagram which shows the basic tree model for this datastore. This is the main store for entities (people or machines) within Olympus. Each entity is keyed on uuid, which is generated by prometheus when the record is added (although the newer, 3g, informatics database, now directly provides uuids). This is organised in a tree structure, such that each entity may have one or more Identities, which in turn may have one or more Account objects (representing a unix account). In practice, most relationships will be one to one (e.g. an entity has one identity, which in turn has one account).

  • IdentityTreeStore (ou=Identities). Identity objects and their account objects are propagated from the entity tree store to a separate read-only tree store for use by conduits.

  • AccountStore (ou=Accounts). Account objects are propagated from the entity tree store to a separate read-only store for use by conduits.

  • GroupStore (ou=Groups). This is populated from the rfe groups file - it maps gid and group name.

  • RoleStore (ou=Roles). Populated from rfe roles files - an entry for each role, with the entitlements and other roles that it possesses.

  • ConduitStore (ou=Conduits). This is where conduits register themselves and store audit results. They can store configuration information here, in their top-level object.

  • ErrorStore (ou=Errors).

  • EventQueue (ou=EventQueue). This is prometheus's event queue. See PrometheusEventQueue for more details.

Note that IdentityTreeStore and AccountStore were originally designed as read-only stores for use by the conduits. As prometheus has developed, the usefulness of these stores has become highly debatable (as an example, the AFSVL conduit cannot use the AccountStore, as planned, because it needs to write home directory information back to the main EntityTreeStore following the creation of a home directory). There is a high probability that these read-only stores will be deprecated and eventually removed, with all conduits being modified to use the EntityTreeStore.

External Datastores

All 'SystemLDAP' stores refer to the LDAP directory used on DICE machines (for rfc 2307 compatibility, etc). They all speak directly to our LDAP master.

  • SystemLDAP::People (ou=People)
  • SystemLDAP::GroupStore (ou=Group)
  • SystemLDAP::NetgroupStore (ou=Netgroup)
  • SystemLDAP::CapabilityStore (ou=Capabilities)

  • KDC - This store speaks directly to our master KDC and is used to manage principals. Currently we only manage primary user principals, but will shortly add management of host-based service principals and additional user principals (e.g. 'user/admin').

  • AFSPTS - This store communicates with the AFS pts database and currently manages user entries only.

  • AFSVL - This store is responsible for managing volumes in the AFS vldb database.

Conduits

Data travels between data stores via conduits, the data flows in and out of prometheus are shown in this diagram.

All conduits should have a unique uuid which identifies them.

Conduits share a consistent interface - exposing the methods 'syncronise' and 'audit', described below.

  • syncronise() - this method, when run, should ensure that data is fully synchronised between datastores, as per the logic in the conduit (this can differ greatly between conduits). This method can return errors, which should be reported by whatever calls the conduit.

  • audit() - this method reports auditable errors (as determined by the logic within the conduit). These typically correlate to changes which would be made if the syncronise() method were subsequently called but can (and do) report other errors. Audit errors are stored in the ConduitStore part of the LDAP directory (see below).

The perl script bin/run-conduit is provided in the prometheus source for running conduits. This is currently called through a wrapper script (/usr/bin/prometheus-run-conduit) to abstract out configuration details. The short names referred to below are those used in this wrapper script - running /usr/bin/prometheus-run-conduit -l on the prometheus server gives a list of all the short names supported by this script).

The conduits run (specifically, their syncronise() methods are called) in a loop on the prometheus server. It runs from Monday-Friday, from 09:30am until approximately 22:00.

All short names used to describe conduits are those used by /usr/bin/prometheus-run-conduit.

Import Conduits

The following conduits, generally speaking, are import conduits, meaning they are used to put information into Olympus.

  • infdb - now defunct

  • infdb3g - Populates the prometheus entity tree store from the School database. We get username, name, upstream (primary) roles, email, location and telephone number information from this store. This is our master for whether a user has an entity object and upstream roles they possess. It thus determines their right to an account. The infdb and infdb2g conduits used to coexist in an uneasy state, but this is no longer the situation now infdb3g is the undisputed master.

  • hosts - Imports host information from a specially constructed LCFG profile (as used previously by lcfg2ldap) into the entity tree store. We currently populate hostname, ip addresses and aliases. There is support for importing roles/capabilities, although this isn't currently used.

  • roleexpander - This conduit expands any roles (both upstream roles and additional roles) and additional entitlements that an entity possesses into a flat list of entitlements and populates upstreamentitlements on the entity object from this. It uses protectedentitlements to manage lifecycle.

  • populateidentity - For every entity in the entity tree store, create an identity for them if they have the triggering entitlement defined by the configuration class's trigger attribute (this is prometheus/localIdentity). This conduit creates identities of the form username@INF.ED.AC.UK

  • populateaccount - For every identity in the entity tree store, add a Unix account object for it, if the identity is marked as being the primary identity for that user (possesses the 'primaryIdentity' flag). Our conduit provides the following default values: gid=10000, shell=/bin/bash

  • isuid - Uses the IS UID server to import uids into account objects in the entity tree store. The syncronise mechanism will only update accounts which have empty uid values - audit will complain about all accounts which have mismatches.

  • mergeidentity - This conduit takes the existing entity tree under ou=Entities (the tree is of the form entity->identity->account) and propagates identity->account tree to ou=Identities.

  • mergeaccount - This conduit takes the existing entity tree and propagates account objects to ou=Accounts

  • kdc-auth-stats - This conduit queries the master KDC for authentication statistics (last successful authentication, last failed authentication and last password change dates) and updates Identity objects. See PrometheusLifecycle#Inactivity_suspension for more detail on how this data is used.

Export Conduits

The following conduits use the data in Prometheus to modify data in external stores.

  • kdc-master - Uses the identity objects in the entity tree store to populate the master KDC. This conduit creates new principals with a random password and with the DISALLOW_ALL_TIX attribute set. It also sets the 'initialPassword' flag on the identity object within Prometheus. This prometheus flag and KDC attribute are then both unset when init-passwds is used to generate a new password for the user. The KDC conduit also acts on the presence of a 'disableAccount' flag on the identity object.

  • afspts-master - Uses the account objects in the identity tree store to populate the AFS pts database.

  • afsvol-master - Uses the account objects in the entity tree store to create home directories. It creates the home volume, the mirror volumes, the backup volume and the appropriate mountpoint in the AFS file system. The conduit is provided with partition arguments, from which it will allocate a suitable location for users, based on entitlements for the user/partition and a space-allocation algorithm. When a home directory has been created, this conduit sets the homedirectory attribute on the entity object.

  • systemldapexport-master - Uses the account store in Prometheus to populate ou=People on the LDAP master server.

  • systemldapents-master - This conduit takes information directly from the entity objects in the entity tree store and uses that to create a temporary internal store mapping entitlements to the list of entities who possess that entitlement. This is then used to populate ou=Group, ou=Netgroup and ou=Capabilities on the LDAP master.

Other Conduits

There are other conduits which are neither import nor export, but instead act in some way on the data held within prometheus.

  • mailnewusers - this conduit mails new student users with details of their new account; it is run from cron

  • lifecycle-processing - this conduit acts on matters of lifecycle, see PrometheusLifecycle for more details

Conduit Loop

The conduits on the prometheus master run in an almost continual loop. The loop runs only on Mon-Fri, between the hours of 09:30 and approximately 22:00/23:00 (this is because the school database runs various synchronisation jobs overnight and its data is not considered stable until 09:30).

The conduits run in the following order:

infdb3g hosts roleexpander populateidentity populateaccount isuid mergeidentity kdc-master afspts-master afsvol-master mergeaccount systemldapexport-master roleexpander systemldapents-master

(Note that the mergeaccount conduit is run after the afs volume conduit, to allow the homedirectory attribute to be propagated to the account tree, which is used by they systemldapexport-master conduit)

(Also note that the roleexpander conduit is run twice - the second run is so that any role/entitlement changes which have been made to an entity are picked up by the subsequent systemldapents-master conduit, otherwise they would have to wait for another loop before being propagated)

Conduits on Other Machines

In addition to the conduit loop that runs continually on the prometheus server, the following conduits run on other machines:

  • roles - This conduit runs on the rfe server every time a change is made to a role. It populates ou=Roles in prometheus. See /etc/rfe.conf and /usr/sbin/prometheus-roles-sync.

  • groups - This conduit runs on the rfe server every time a change is made to the groups rfe map. It populates ou=Groups in prometheus. See /etc/rfe.conf and /usr/bin/prometheus-group-sync.

Note that the wrapper scripts detailed above (/usr/sbin/prometheus-roles-sync and /usr/bin/prometheus-group-sync) are provided by the prometheus-rfe-scripts RPM.

Error Reporting

Conduits can return prometheus errors. The use of these has changed since prometheus was first designed and the term "error" is probably inaccurate. These are now used to report anything exceptional which we might need to know about. Any errors generated can be mailed using the --mailerrors argument to bin/run-conduit (exposed through -e in prometheus-run-conduit).

Prometheus Objects and Object Trees: A Closer Look

As described above, the EntityTreeStore is central to much of prometheus's operation. This diagram shows the basic relationships between the different objects in the structure.

This section will provide a few more details on how this works in a few key areas.

Roles/Entitlements

Attributes

Each Entity, Identity and Account object can possess the following attributes: upstreamroles, upstreamentitlements, additionalroles, additionalentitlements, protectedentitlements.

Entities and Role Expansion

The role/entitlements attributes at the Entity level (i.e. at the top level of the tree) can be described as follows:

  • upstreamroles - This is the list of roles obtained directly from the school database.
  • additionalroles - This is a list of additional roles added (probably manually) to the object.
  • additionalentitlements - This is a list of additional entitlements added (probably manually) to the object.
  • upstreamentitlements - This attribute is not ideally named. It is the complete list of entitlements possessed by an entity.
  • protectedentitlements - This attribute is used to maintain entitlements when a user's account has expired is in a grace period.

The upstreamentitlements attribute on an Entity object is generated by the RoleExpander conduit using the following logic (taken from the conduit's documentation):

This conduit iterates through the EntityTreeStore and expands the
roles for each entity.  It does this by recursively generating lists
of entitlements for each included role (both 'upstreamroles' and
'additionalroles'), adding any entitlements defined in
'additionalentitlements' and finally removing any negative
entitlements.  The final expanded list of entitlements is put in
'upstreamentitlements'

Note that the 'protectedentitlements' attribute is used to maintain state of an entity's collected entitlements. See PrometheusLifecycle for details of how this works.

Child Objects and Role/Entitlement Inheritance

To allow a more finely-grained approach for child objects (identities and accounts) in the entity tree store, it was necessary to implement an entitlement inheritance model. This is described below:

  • An object silently inherits the upstream entitlements of the parent object, unless it explicitly has upstream entitlements of its own.
  • An object can augment inherited entitlements by defining additional entitlements
  • All roles attributes are inherited directly from the Entity object (note this has little bearing on functionality, as roles are just used as providers of entitlements).

Ownership

Prometheus objects can have a concept of ownership. This allows conduits to register ownership of any objects, which will influence how other conduits can interact with an object. This works in the following way:

  • A conduit constructs a datastore with the owner attribute set to the conduit's uuid. This makes the store an "owned store". A datastore constructed with no owner attribute set is an "unowned store".
  • An object can have more than one owner.
  • A conduit adding an object to an owned store will set the ownership of that object to the conduit's uuid.
  • A conduit modifying an object in an owned store will add its uuid to the list of object owners.
  • A conduit deleting an object in an owned store will remove its uuid from the list of object owners. If this leads to an empty ownership list, then the object is actually deleted.
  • A conduit is permitted to delete any object from an unowned store, irrespective of the object's ownership (it acts as a super-user in this respect).

Child Objects and Ownership Inheritance

The following rules apply to ownership inheritance/management for child objects

  • Calling update on an object sets ownership for all children in the tree as well as the parent.
  • New children inherit the ownership of their immediate parent.
  • Deletion of children adheres to defined deletion rules - child object remains until all owning stores have deleted.

Flags

Prometheus uses a series of flags to influence its behaviour. These can "belong" to a conduit (in which case they are stored as "conduit-uuid:flagname"). These are typically stored against an identity object. The following flags are currently in use (conduit ownership in parentheses):

  • primaryIdentity - this is used to indicate that an identity is the user's primary one; it is not owned by any conduit
  • mailSent (mailnewusers) - indicates that an email has been sent to notify of a new account
  • initialPassword (kdc-master) - indicates that this identity has a newly created principal in the KDC and can have its password set using init-passwds
  • disableAccount (kdc-master) - indicates that this identity should be disabled (by setting the DISALLOW_ALL_TIX attribute on the KDC principal)
  • accountLockout (kdc-master) - used with disableAccount, above, to lock an account (see PrometheusClientSide#Using_theogony_to_lockout_an_acc)
  • requirePasswordChange (kdc-master) - indicates that this identity should be disabled (see above) until the password has been changed using the password portal - this flag is rarely used now
  • expiryMailSent (lifecycle-processing) - indicates that this user has been sent the email to indicate their account has expired and they are now in a grace period
  • inactivitySuspension (lifecycle-processing) - indicates that a user's account has been suspended for reasons of inactivity - if this flag is present, but the disableAccount flag is not, then the account has been re-enabled but the lifecycle-processing conduit has not yet removed this flag (this is a two-stage process, see: PrometheusLifecycle#Inactivity_suspension)
  • inactivityMailSent (lifecycle-processing) - indicates that this user has been sent the email to indicate their account has been suspended due to inactivity
  • noLifecycleProcessing (lifecycle-processing) - this tells the lifecycle-processing conduit not to do anything (send email, disable) to this user.

Comments

Comments can be added to prometheus objects. This is documented here:

PrometheusComments

Event Queue

Prometheus has an event queue, which allows a user (or application) to request the running of a particular conduit (e.g. to speed up propagation of data).

It is documented here:

PrometheusEventQueue


-- TobyBlake - 06 Mar 2019

Topic revision: r23 - 06 Mar 2019 - 14:24:18 - TobyBlake
 
This site is powered by the TWiki collaboration platformCopyright © by the contributing authors. All material on this collaboration platform is the property of the contributing authors.
Ideas, requests, problems regarding TWiki? Send feedback
This Wiki uses Cookies