Working with Hypatia
Hypatia is the name of the School Database service. It encapsulates the data model,
TheonUI desktops,
Incoming framework, legacy conduits and framework, Portal Conduits and framework.
Conduits, associated frameworks and the Incoming framework are specific to Hypatia. The data model
(and associated
TheonUI desktops and
TheonCoupler feed definitions) are defined under a Theon
ModelLibrary .
Getting a Copy
Hypatia is held in a Git repository. While working with this can be superficially similar to Subversion, there
are differences to be aware of. In particular you work with your own copy of the entire repository. Changes
you make are local to your own repository and must be specifically pushed back to the central repository.
To get a copy on a DICE machine you can do:
git clone /afs/inf.ed.ac.uk/group/git-repositories/timc/hypatia
To get a copy from a non-DICE machine and from anywhere you can do:
git clone git://afsgit.inf.ed.ac.uk/timc/hypatia.git
Either way after cloning you will end up with a
hypatia directory containing the full repository and
all history. This is about 0.5GB. If you don't need to refer to history you can do a shallow clone instead.
git clone --depth 1 file:///afs/inf.ed.ac.uk/group/git-repositories/timc/hypatia
This will only contain the most recent commit, and will be tiny by comparison (a few MB). You can still carry
out a lot of common operations in a shallow clone (such as making changes and pushing them back).
Making Changes to Portal Conduits
The Portal Framework is a derivative of the
ConduitsFramework, for more in depth detail refer to that.
The most common change is to a Portal conduit.
Running Conduits
Typically you will need to set three environment variables to use the commands below; ''how'' to set them is explained in detail below but in summary you will need to set at least one of:
*
$CONDUITS
- conduit source directory, containing {{{conduits}}}, {{{includes}}}, etc.
*
$MY_CONDUITS
- effectively the output root directory
*
$DB_HOSTNAME
- defaults to localhost
read on for full details.
The portal framework can be used on any machine with the
dice/options/hypatia-portal-server.h
header. By default it creates a working directory in
/dev/shm/portal
; you can change this by ensuring the environment variable
MY_CONDUITS
is appropriately set beforehand.
By default this working directory is not protected, you should do so as many generated portal files are for restricted viewing. To initialise your working directory do the following.
portal --init
Assuming you have the relevant permissions you can then run any portal conduit using the command below.
portal RUN TPNNN_Conduit_Name
NOTE: use the runconduit method if you have trouble with the above...
Here
TPNNN_Conduit_Name
is the name of the portal conduit, to list all available portal conduits do
ls /usr/lib/hypatia/conduits/portal/conduits/TP*
The generated output of the portal conduit (as displayed when run above) will be relative to
/dev/shm/portal
(or
$MY_CONDUITS
). You can view the generated output with
w3m
,
less
or point a local browser at
file:///dev/shm/portal/...
.
If you explicitly set
MY_CONDUITS
to {{{/dev/shm/portal}}} (or a custom path) then you can also use {{{runconduit}}} to run and generate portal conduits.
Unless you are running on an infdb server, you will need to set the environment variable
DB_HOSTNAME
to the hostname of the School database server (or a development server).
Having done all of this you can then run your conduit using the
portal
invocation above.
Using runconduit
As an alternative to the
portal
command you can use:
runconduit TPNNN_Conduit_Name
The only real benefit of this is that it supports the use of the
-d
debug option (after the portal conduit argument) to display the expanded SQL, etc. It also bypasses some of the portal framework wrapping.
To use this command you should have
$MY_CONDUITS
set to
~/.conduits
for running
portal --init
*and* runconduit
; you will also need to set up a symbolic link:
ln -s /dev/shm/portal/holding/conduits.txt $MY_CONDUITS/holding/
See Gurgle's documentation for details of the
-d
argument output.
Changing Conduits
The following is done from within your clone directory.
cd portal/conduits/conduits
${EDITOR} TPnnn_A_B_C
# test your changes by using "runconduit" or "portal RUN" above
# once working stage your changes - this adds them to the "commit" list but does not actually commit them
git add TPnnn_A_B_C
# commit all staged changes
git commit
At this point the change to the conduit has been made and committed. But this change is only in your
cloned copy. It needs to be pushed back into the central repository.
git push origin master
If changes have been made in the central repository you will get an error, which effectively means
you need to update your local repository to match, resolve merge conflicts (if any) and push again.
git pull --rebase origin master
Deploying Changes to Portal Conduits
This is pretty much as before, just use
makerpm which has been adapted for Git.
cd ../..
./makerpm
Note that you now get an abbreviated SHA1 code as the release rather than a revision number
as with Subversion. You can double-check changes between RPMs with
git diff <old sha1> [new sha1]
RPMs made using this method should be treated as a pre-release. You can make a formal release
using the Theon toolkit. The default is to update the minor number, in a lot of cases you will
want to be updating the bugfix number instead, see the steps below.
ttkb release upfix
or if it really is a new feature then just
ttkb release
As before these changes are only made in the local repository and need to be pushed.
git push origin master
The packages for the new release can be built as below.
ttkb package into /tmp
These can then be passed to
pkgsubmit or
pkgforge as usual.
Changing the Model
The Theon
ModelLibrary for Hypatia is in
library/hypatia. Refer to the Theon Book
for manipulating and distributing the content in here.
--
TimColles - 04 Dec 2017