Working with Hypatia

Hypatia is the name of the School Database service. It encapsulates the data model, TheonUI desktops, Incoming framework, legacy conduits and framework, Portal Conduits and framework. Conduits, associated frameworks and the Incoming framework are specific to Hypatia. The data model (and associated TheonUI desktops and TheonCoupler feed definitions) are defined under a Theon ModelLibrary .

Getting a Copy

Hypatia is held in a Git repository. While working with this can be superficially similar to Subversion, there are differences to be aware of. In particular you work with your own copy of the entire repository. Changes you make are local to your own repository and must be specifically pushed back to the central repository.

To get a copy on a DICE machine you can do:

git clone /afs/inf.ed.ac.uk/group/git-repositories/timc/hypatia

To get a copy from a non-DICE machine and from anywhere you can do:

git clone git://afsgit.inf.ed.ac.uk/timc/hypatia.git

Either way after cloning you will end up with a hypatia directory containing the full repository and all history. This is about 0.5GB. If you don't need to refer to history you can do a shallow clone instead.

git clone --depth 1 file:///afs/inf.ed.ac.uk/group/git-repositories/timc/hypatia

This will only contain the most recent commit, and will be tiny by comparison (a few MB). You can still carry out a lot of common operations in a shallow clone (such as making changes and pushing them back).

Making Changes to Portal Conduits

The Portal Framework is a derivative of the ConduitsFramework, for more in depth detail refer to that. The most common change is to a Portal conduit.

Running Conduits

Typically you will need to set three environment variables to use the commands below; ''how'' to set them is explained in detail below but in summary you will need to set at least one of: * $CONDUITS - conduit source directory, containing {{{conduits}}}, {{{includes}}}, etc. * $MY_CONDUITS - effectively the output root directory * $DB_HOSTNAME - defaults to localhost read on for full details.

The portal framework can be used on any machine with the dice/options/hypatia-portal-server.h header. By default it creates a working directory in /dev/shm/portal; you can change this by ensuring the environment variable MY_CONDUITS is appropriately set beforehand. By default this working directory is not protected, you should do so as many generated portal files are for restricted viewing. To initialise your working directory do the following.

portal --init

Assuming you have the relevant permissions you can then run any portal conduit using the command below.

portal RUN TPNNN_Conduit_Name

NOTE: use the runconduit method if you have trouble with the above...

Here TPNNN_Conduit_Name is the name of the portal conduit, to list all available portal conduits do

ls /usr/lib/hypatia/conduits/portal/conduits/TP*

The generated output of the portal conduit (as displayed when run above) will be relative to /dev/shm/portal (or $MY_CONDUITS). You can view the generated output with w3m, less or point a local browser at file:///dev/shm/portal/....

If you explicitly set MY_CONDUITS to {{{/dev/shm/portal}}} (or a custom path) then you can also use {{{runconduit}}} to run and generate portal conduits.

Unless you are running on an infdb server, you will need to set the environment variable DB_HOSTNAME to the hostname of the School database server (or a development server).

Having done all of this you can then run your conduit using the portal invocation above.

Using runconduit

As an alternative to the portal command you can use:

runconduit TPNNN_Conduit_Name

The only real benefit of this is that it supports the use of the -d debug option (after the portal conduit argument) to display the expanded SQL, etc. It also bypasses some of the portal framework wrapping.

To use this command you should have $MY_CONDUITS set to ~/.conduits for running portal --init *and* runconduit; you will also need to set up a symbolic link:

ln -s /dev/shm/portal/holding/conduits.txt $MY_CONDUITS/holding/

See Gurgle's documentation for details of the -d argument output.

Changing Conduits

The following is done from within your clone directory.

cd portal/conduits/conduits
${EDITOR} TPnnn_A_B_C
# test your changes by using "runconduit" or "portal RUN" above
# once working stage your changes - this adds them to the "commit" list but does not actually commit them
git add TPnnn_A_B_C
# commit all staged changes
git commit

At this point the change to the conduit has been made and committed. But this change is only in your cloned copy. It needs to be pushed back into the central repository.

git push origin master

If changes have been made in the central repository you will get an error, which effectively means you need to update your local repository to match, resolve merge conflicts (if any) and push again.

git pull --rebase origin master

Deploying Changes to Portal Conduits

This is pretty much as before, just use makerpm which has been adapted for Git.

cd ../..
./makerpm

Note that you now get an abbreviated SHA1 code as the release rather than a revision number as with Subversion. You can double-check changes between RPMs with

git diff <old sha1> [new sha1]

RPMs made using this method should be treated as a pre-release. You can make a formal release using the Theon toolkit. The default is to update the minor number, in a lot of cases you will want to be updating the bugfix number instead, see the steps below.

ttkb release upfix

or if it really is a new feature then just

ttkb release

As before these changes are only made in the local repository and need to be pushed.

git push origin master

The packages for the new release can be built as below.

ttkb package into /tmp

These can then be passed to pkgsubmit or pkgforge as usual.

Changing the Model

The Theon ModelLibrary for Hypatia is in library/hypatia. Refer to the Theon Book for manipulating and distributing the content in here.

-- TimColles - 04 Dec 2017