Class and Items

A Class defines a particular class (or type) of data that will be stored in the database. A class comprises one or more properties, which gives the information about the class items.

The actual data entered into the database, using class.create(), are called items. They have a special immutable property called 'id'. We sometimes refer to this as the itemid.

Properties

A Class is comprised of one or more properties of the following types:

• String properties are for storing arbitrary-length strings.
• Password properties are for storing encoded arbitrary-length strings. The default encoding is defined on the roundup.password.Password class.
• Date properties store date-and-time stamps. Their values are Timestamp objects.
• Number properties store numeric values.
• Boolean properties store on/off, yes/no, true/false values.
• A Link property refers to a single other item selected from a specified class. The class is part of the property; the value is an integer, the id of the chosen item.
• A Multilink property refers to possibly many items in a specified class. The value is a list of integers.

All Classes automatically have a number of properties by default:

creator

Link to the user that created the item.

creation

Date the item was created.

actor

Link to the user that last modified the item.

activity

Date the item was last modified.

FileClass

FileClasses save their “content” attribute off in a separate file from the rest of the database. This reduces the number of large entries in the database, which generally makes databases more efficient, and also allows us to use command-line tools to operate on the files. They are stored in the files sub-directory of the 'db' directory in your tracker. FileClasses also have a “type” attribute to store the MIME type of the file.

IssueClass

IssueClasses automatically include the “messages”, “files”, “nosy”, and “superseder” properties.

The messages and files properties list the links to the messages and files related to the issue. The nosy property is a list of links to users who wish to be informed of changes to the issue - they get “CC’ed” e-mails when messages are sent to or generated by the issue. The nosy reactor (in the 'detectors' directory) handles this action. The superseder link indicates an issue which has superseded this one.

They also have the dynamically generated “creation”, “activity” and “creator” properties.

The value of the “creation” property is the date when an item was created, and the value of the “activity” property is the date when any property on the item was last edited (equivalently, these are the dates on the first and last records in the item’s journal). The “creator” property holds a link to the user that created the issue.

setkey(property)

Select a String property of the class to be the key property. The key property must be unique, and allows references to the items in the class by the content of the key property. That is, we can refer to users by their username: for example, let’s say that there’s an issue in roundup, issue 23. There’s also a user, richard, who happens to be user 2. To assign an issue to him, we could do either of:

roundup-admin set issue23 assignedto=2

or:

roundup-admin set issue23 assignedto=richard

Note, the same thing can be done in the web and e-mail interfaces.

setlabelprop(property)

Select a property of the class to be the label property. The label property is used whereever an item should be uniquely identified, e.g., when displaying a link to an item. If setlabelprop is not specified for a class, the following values are tried for the label:

• the key of the class (see the setkey(property) section above)
• the “name” property
• the “title” property
• the first property from the sorted property name list

So in most cases you can get away without specifying setlabelprop explicitly.

setorderprop(property)

Select a property of the class to be the order property. The order property is used whenever using a default sort order for the class, e.g., when grouping or sorting class A by a link to class B in the user interface, the order property of class B is used for sorting. If setorderprop is not specified for a class, the following values are tried for the order property:

• the property named “order”
• the label property (see setlabelprop(property) above)

So in most cases you can get away without specifying setorderprop explicitly.

create(information)

Create an item in the database. This is generally used to create items in the “definitional” classes like “priority” and “status”.

A note about ordering

When we sort items in the hyperdb, we use one of a number of methods, depending on the properties being sorted on:

1. If it’s a String, Number, Date or Interval property, we just sort the scalar value of the property. Strings are sorted case-sensitively.
2. If it’s a Link property, we sort by either the linked item’s “order” property (if it has one) or the linked item’s “id”.
3. Mulitlinks sort similar to #2, but we start with the first Multilink list item, and if they’re the same, we sort by the second item, and so on.

Note that if an “order” property is defined on a Class that is used for sorting, all items of that Class must have a value against the “order” property, or sorting will result in random ordering.

Adding a new Permission

When adding a new Permission, you will need to:

1. add it to your tracker’s schema.py so it is created, using security.addPermission, for example:

   self.security.addPermission(name="View", klass='frozzle',
       description="User is allowed to access frozzles")
   

   will set up a new “View” permission on the Class “frozzle”.

2. enable it for the Roles that should have it (verify with “roundup-admin security“)

3. add it to the relevant HTML interface templates

4. add it to the appropriate xxxPermission methods on in your tracker interfaces module

The addPermission method takes a couple of optional parameters:

properties

A sequence of property names that are the only properties to apply the new Permission to (eg. ... klass='user', properties=('name', 'email') ...)

check

A function to be execute which returns boolean determining whether the Permission is allowed. The function has the signature check(db, userid, itemid) where db is a handle on the open database, userid is the user attempting access and itemid is the specific item being accessed.

Example Scenarios

See the examples section for longer examples of customisation.

anonymous access through the e-mail gateway

Give the “anonymous” user the “Email Access”, (“Edit”, “issue”) and (“Create”, “msg”) Permissions but do not not give them the (“Create”, “user”) Permission. This means that when an unknown user sends email into the tracker, they’re automatically logged in as “anonymous”. Since they don’t have the (“Create”, “user”) Permission, they won’t be automatically registered, but since “anonymous” has permission to use the gateway, they’ll still be able to submit issues. Note that the Sender information - their email address - will not be available - they’re anonymous.

automatic registration of users in the e-mail gateway

By giving the “anonymous” user the (“Register”, “user”) Permission, any unidentified user will automatically be registered with the tracker (with no password, so they won’t be able to log in through the web until an admin sets their password). By default new Roundup trackers don’t allow this as it opens them up to spam. It may be enabled by uncommenting the appropriate addPermissionToRole in your tracker’s schema.py file. The new user is given the Roles list defined in the “new_email_user_roles” config variable.

only developers may be assigned issues

Create a new Permission called “Fixer” for the “issue” class. Create a new Role “Developer” which has that Permission, and assign that to the appropriate users. Filter the list of users available in the assignedto list to include only those users. Enforce the Permission with an auditor. See the example restricting the list of users that are assignable to a task.

only managers may sign off issues as complete

Create a new Permission called “Closer” for the “issue” class. Create a new Role “Manager” which has that Permission, and assign that to the appropriate users. In your web interface, only display the “resolved” issue state option when the user has the “Closer” Permissions. Enforce the Permission with an auditor. This is very similar to the previous example, except that the web interface check would look like:

<option tal:condition="python:request.user.hasPermission('Closer')"
        value="resolved">Resolved</option>

don’t give web access to users who register through email

Create a new Role called “Email User” which has all the Permissions of the normal “User” Role minus the “Web Access” Permission. This will allow users to send in emails to the tracker, but not access the web interface.

let some users edit the details of all users

Create a new Role called “User Admin” which has the Permission for editing users:

db.security.addRole(name='User Admin', description='Managing users')
p = db.security.getPermission('Edit', 'user')
db.security.addPermissionToRole('User Admin', p)

and assign the Role to the users who need the permission.

Templating engines

Since version 1.4.20 Roundup supports two templating engines: the original Template Attribute Language (TAL) engine from Zope and the standalone Chameleon templating engine. Chameleon is intended as a replacement for the original TAL engine, and supports the same syntax, but they are not 100% compatible. The major (and most likely the only) incompatibility is the default expression type being python: instead of path:. See also “Incompatibilities and differences” section of Chameleon documentation.

NOTE1: For historical reasons, examples given below assumes path expression as default expression type. With Chameleon you have to manually resolve the path expressions. A Chameleon-based, z3c.pt, that is fully compatible with the old TAL implementation, is planned to be included in a future release.

NOTE2: As of 1.4.20 Chameleon support is highly experimental and not recommended for production use.

Basic Templating Actions

Roundup’s templates consist of special attributes on the HTML tags. These attributes form the Template Attribute Language, or TAL. The basic TAL commands are:

tal:define=”variable expression; variable expression; ...”

Define a new variable that is local to this tag and its contents. For example:

<html tal:define="title request/description">
 <head><title tal:content="title"></title></head>
</html>

In this example, the variable “title” is defined as the result of the expression “request/description”. The “tal:content” command inside the <html> tag may then use the “title” variable.

tal:condition=”expression”

Only keep this tag and its contents if the expression is true. For example:

<p tal:condition="python:request.user.hasPermission('View', 'issue')">
 Display some issue information.
</p>

In the example, the <p> tag and its contents are only displayed if the user has the “View” permission for issues. We consider the number zero, a blank string, an empty list, and the built-in variable nothing to be false values. Nearly every other value is true, including non-zero numbers, and strings with anything in them (even spaces!).

tal:repeat=”variable expression”

Repeat this tag and its contents for each element of the sequence that the expression returns, defining a new local variable and a special “repeat” variable for each element. For example:

<tr tal:repeat="u user/list">
 <td tal:content="u/id"></td>
 <td tal:content="u/username"></td>
 <td tal:content="u/realname"></td>
</tr>

The example would iterate over the sequence of users returned by “user/list” and define the local variable “u” for each entry. Using the repeat command creates a new variable called “repeat” which you may access to gather information about the iteration. See the section below on the repeat variable.

tal:replace=”expression”

Replace this tag with the result of the expression. For example:

<span tal:replace="request/user/realname" />

The example would replace the <span> tag and its contents with the user’s realname. If the user’s realname was “Bruce”, then the resultant output would be “Bruce”.

tal:content=”expression”

Replace the contents of this tag with the result of the expression. For example:

<span tal:content="request/user/realname">user's name appears here
</span>

The example would replace the contents of the <span> tag with the user’s realname. If the user’s realname was “Bruce” then the resultant output would be “<span>Bruce</span>”.

tal:attributes=”attribute expression; attribute expression; ...”

Set attributes on this tag to the results of expressions. For example:

<a tal:attributes="href string:user${request/user/id}">My Details</a>

In the example, the “href” attribute of the <a> tag is set to the value of the “string:user${request/user/id}” expression, which will be something like “user123”.

tal:omit-tag=”expression”

Remove this tag (but not its contents) if the expression is true. For example:

<span tal:omit-tag="python:1">Hello, world!</span>

would result in output of:

Hello, world!

Note that the commands on a given tag are evaulated in the order above, so define comes before condition, and so on.

Additionally, you may include tags such as <tal:block>, which are removed from output. Its content is kept, but the tag itself is not (so don’t go using any “tal:attributes” commands on it). This is useful for making arbitrary blocks of HTML conditional or repeatable (very handy for repeating multiple table rows, which would othewise require an illegal tag placement to effect the repeat).

Templating Expressions

Templating Expressions are covered by Template Attribute Language Expression Syntax, or TALES. The expressions you may use in the attribute values may be one of the following forms:

Path Expressions - eg. item/status/checklist

These are object attribute / item accesses. Roughly speaking, the path item/status/checklist is broken into parts item, status and checklist. The item part is the root of the expression. We then look for a status attribute on item, or failing that, a status item (as in item['status']). If that fails, the path expression fails. When we get to the end, the object we’re left with is evaluated to get a string - if it is a method, it is called; if it is an object, it is stringified. Path expressions may have an optional path: prefix, but they are the default expression type, so it’s not necessary.

If an expression evaluates to default, then the expression is “cancelled” - whatever HTML already exists in the template will remain (tag content in the case of tal:content, attributes in the case of tal:attributes).

If an expression evaluates to nothing then the target of the expression is removed (tag content in the case of tal:content, attributes in the case of tal:attributes and the tag itself in the case of tal:replace).

If an element in the path may not exist, then you can use the | operator in the expression to provide an alternative. So, the expression request/form/foo/value | default would simply leave the current HTML in place if the “foo” form variable doesn’t exist.

You may use the python function path, as in path("item/status"), to embed path expressions in Python expressions.

String Expressions - eg. string:hello ${user/name}

These expressions are simple string interpolations - though they can be just plain strings with no interpolation if you want. The expression in the ${ ... } is just a path expression as above.

Python Expressions - eg. python: 1+1

These expressions give the full power of Python. All the “root level” variables are available, so python:item.status.checklist() would be equivalent to item/status/checklist, assuming that checklist is a method.

Modifiers:

structure - eg. structure python:msg.content.plain(hyperlink=1)

The result of expressions are normally escaped to be safe for HTML display (all “<”, “>” and “&” are turned into special entities). The structure expression modifier turns off this escaping - the result of the expression is now assumed to be HTML, which is passed to the web browser for rendering.

not: - eg. not:python:1=1

This simply inverts the logical true/false value of another expression.

Template Macros

Macros are used in Roundup to save us from repeating the same common page stuctures over and over. The most common (and probably only) macro you’ll use is the “icing” macro defined in the “page” template.

Macros are generated and used inside your templates using special attributes similar to the basic templating actions. In this case, though, the attributes belong to the Macro Expansion Template Attribute Language, or METAL. The macro commands are:

metal:define-macro=”macro name”

Define that the tag and its contents are now a macro that may be inserted into other templates using the use-macro command. For example:

<html metal:define-macro="page">
 ...
</html>

defines a macro called “page” using the <html> tag and its contents. Once defined, macros are stored on the template they’re defined on in the macros attribute. You can access them later on through the templates variable, eg. the most common templates/page/macros/icing to access the “page” macro of the “page” template.

metal:use-macro=”path expression”

Use a macro, which is identified by the path expression (see above). This will replace the current tag with the identified macro contents. For example:

<tal:block metal:use-macro="templates/page/macros/icing">
 ...
</tal:block>

will replace the tag and its contents with the "page" macro of the
"page" template.

metal:define-slot=”slot name” and metal:fill-slot=”slot name”

To define dynamic parts of the macro, you define “slots” which may be filled when the macro is used with a use-macro command. For example, the templates/page/macros/icing macro defines a slot like so:

<title metal:define-slot="head_title">title goes here</title>

In your use-macro command, you may now use a fill-slot command like this:

<title metal:fill-slot="head_title">My Title</title>

where the tag that fills the slot completely replaces the one defined as the slot in the macro.

Note that you may not mix METAL and TAL commands on the same tag, but TAL commands may be used freely inside METAL-using tags (so your fill-slots tags may have all manner of TAL inside them).

The context variable

The context variable is one of three things based on the current context (see determining web context for how we figure this out):

1. if we’re looking at a “home” page, then it’s None
2. if we’re looking at a specific hyperdb class, it’s a hyperdb class wrapper.
3. if we’re looking at a specific hyperdb item, it’s a hyperdb item wrapper.

If the context is not None, we can access the properties of the class or item. The only real difference between cases 2 and 3 above are:

1. the properties may have a real value behind them, and this will appear if the property is displayed through context/property or context/property/field.
2. the context’s “id” property will be a false value in the second case, but a real, or true value in the third. Thus we can determine whether we’re looking at a real item from the hyperdb by testing “context/id”.

python:context['list']

hasPermission(self, permission, [classname=],
    [property=], [itemid=])

hasRole(self, rolename)

python:context['journal']

"structure msg/content/hyperlinked"

python:context.creation.local(10)

python:context.activity.pretty('%Y-%m-%d')

<span tal:replace="structure context/due/popcal" />

<span tal:replace="structure context/due/field" />
<span tal:replace="structure context/due/popcal" />

<span tal:replace="structure context/status/menu" />

<span tal:replace="python:context.status.menu(order='+name",
                      value='chatting',
                      filterspec={'status': '1,2,3,4'}" />

python:context.files.sorted('creation')

context/title/field

The request variable

This is implemented by the roundup.cgi.templating.HTMLRequest class.

The request variable is packed with information about the current request.

Index page specific variables (indexing arguments)

There are several methods available on the request variable:

request/form/name/value

python:request.form['name'].value

The db variable

This is implemented by the roundup.cgi.templating.HTMLDatabase class.

Allows access to all hyperdb classes as attributes of this variable. If you want access to the “user” class, for example, you would use:

db/user
python:db.user

Also, the current id of the current user is available as db.getuid(). This isn’t so useful in templates (where you have request/user), but it can be useful in detectors or interfaces.

The access results in a hyperdb class wrapper.

The templates variable

This was implemented by the roundup.cgi.templating.Templates class before 1.4.20. In later versions it is the instance of appropriate template engine loader class.

This variable is used to access other templates in expressions and template macros. It doesn’t have any useful methods defined. The templates can be accessed using the following path expression:

templates/name

or the python expression:

templates[name]

where “name” is the name of the template you wish to access. The template has one useful attribute, namely “macros”. To access a specific macro (called “macro_name”), use the path expression:

templates/name/macros/macro_name

or the python expression:

templates[name].macros[macro_name]

The repeat variable

The repeat variable holds an entry for each active iteration. That is, if you have a tal:repeat="user db/users" command, then there will be a repeat variable entry called “user”. This may be accessed as either:

repeat/user
python:repeat['user']

The “user” entry has a number of methods available for information:

The utils variable

This is implemented by the roundup.cgi.templating.TemplatingUtils class, but it may be extended as described below.

You may add additional utility methods by writing them in your tracker extensions directory and registering them with the templating system using instance.registerUtil (see adding a time log to your issues for an example of this).

python:utils.Batch(sequence, size, start, end=0, orphan=0,
overlap=0)

request/batch

<table class="otherinfo">
 <tr><th colspan="4" class="header">Existing Keywords</th></tr>
 <tr tal:define="keywords db/keyword/list"
     tal:repeat="start python:range(0, len(keywords), 4)">
  <td tal:define="batch python:utils.Batch(keywords, 4, start)"
      tal:repeat="keyword batch" tal:content="keyword/name">
      keyword here</td>
 </tr>
</table>

Translations

Should you wish to enable multiple languages in template content that you create you’ll need to add new locale files in the tracker home under a locale directory. Use the instructions in the developer's guide to create the locale files.

Index View Specifiers

An index view specifier (URL fragment) looks like this (whitespace has been added for clarity):

/issue?status=unread,in-progress,resolved&
    keyword=security,ui&
    @group=priority,-status&
    @sort=-activity&
    @filters=status,keyword&
    @columns=title,status,fixer

The index view is determined by two parts of the specifier: the layout part and the filter part. The layout part consists of the query parameters that begin with colons, and it determines the way that the properties of selected items are displayed. The filter part consists of all the other query parameters, and it determines the criteria by which items are selected for display. The filter part is interactively manipulated with the form widgets displayed in the filter section. The layout part is interactively manipulated by clicking on the column headings in the table.

The filter part selects the union of the sets of items with values matching any specified Link properties and the intersection of the sets of items with values matching any specified Multilink properties.

The example specifies an index of “issue” items. Only items with a “status” of either “unread” or “in-progress” or “resolved” are displayed, and only items with “keyword” values including both “security” and “ui” are displayed. The items are grouped by priority arranged in ascending order and in descending order by status; and within groups, sorted by activity, arranged in descending order. The filter section shows filters for the “status” and “keyword” properties, and the table includes columns for the “title”, “status”, and “fixer” properties.

Editor Section

The editor section is used to manipulate the item - it may be a static display if the user doesn’t have permission to edit the item.

Here’s an example of a basic editor template (this is the default “classic” template issue item edit form - from the “issue.item.html” template):

<table class="form">
<tr>
 <th>Title</th>
 <td colspan="3" tal:content="structure python:context.title.field(size=60)">title</td>
</tr>

<tr>
 <th>Priority</th>
 <td tal:content="structure context/priority/menu">priority</td>
 <th>Status</th>
 <td tal:content="structure context/status/menu">status</td>
</tr>

<tr>
 <th>Superseder</th>
 <td>
  <span tal:replace="structure python:context.superseder.field(showid=1, size=20)" />
  <span tal:replace="structure python:db.issue.classhelp('id,title')" />
  <span tal:condition="context/superseder">
   <br>View: <span tal:replace="structure python:context.superseder.link(showid=1)" />
  </span>
 </td>
 <th>Nosy List</th>
 <td>
  <span tal:replace="structure context/nosy/field" />
  <span tal:replace="structure python:db.user.classhelp('username,realname,address,phone')" />
 </td>
</tr>

<tr>
 <th>Assigned To</th>
 <td tal:content="structure context/assignedto/menu">
  assignedto menu
 </td>
 <td>&nbsp;</td>
 <td>&nbsp;</td>
</tr>

<tr>
 <th>Change Note</th>
 <td colspan="3">
  <textarea name=":note" wrap="hard" rows="5" cols="60"></textarea>
 </td>
</tr>

<tr>
 <th>File</th>
 <td colspan="3"><input type="file" name=":file" size="40"></td>
</tr>

<tr>
 <td>&nbsp;</td>
 <td colspan="3" tal:content="structure context/submit">
  submit button will go here
 </td>
</tr>
</table>

When a change is submitted, the system automatically generates a message describing the changed properties. As shown in the example, the editor template can use the “:note” and “:file” fields, which are added to the standard changenote message generated by Roundup.

Spool Section

The spool section lists related information like the messages and files of an issue.

TODO

History Section

The final section displayed is the history of the item - its database journal. This is generally generated with the template:

<tal:block tal:replace="structure context/history" />

To be done:

The actual history entries of the item may be accessed for manual templating through the “journal” method of the item:

<tal:block tal:repeat="entry context/journal">
 a journal entry
</tal:block>

where each journal entry is an HTMLJournalEntry.

Define the new action class

Create a new action class in your tracker’s extensions directory, for example myaction.py:

from roundup.cgi.actions import Action

class MyAction(Action):
    def handle(self):
        ''' Perform some action. No return value is required.
        '''

The self.client attribute is an instance of roundup.cgi.client.Client. See the docstring of that class for details of what it can do.

The method will typically check the self.form variable’s contents. It may then:

• add information to self.client.ok_message or self.client.error_message
• change the self.client.template variable to alter what the user will see next
• raise Unauthorised, SendStaticFile, SendFile, NotFound or Redirect exceptions (import them from roundup.cgi.exceptions)

Register the action class

The class is now written, but isn’t available to the user until you register it with the following code appended to your myaction.py file:

def init(instance):
    instance.registerAction('myaction', myActionClass)

This maps the action name “myaction” to the action class we defined.

Use the new action

In your HTML form, add a hidden form element like so:

<input type="hidden" name="@action" value="myaction">

where “myaction” is the name you registered in the previous step.

Actions may return content to the user

Actions generally perform some database manipulation and then pass control on to the rendering of a template in the current context (see Determining web context for how that works.) Some actions will want to generate the actual content returned to the user. Action methods may return their own content string to be displayed to the user, overriding the templating step. In this situation, we assume that the content is HTML by default. You may override the content type indicated to the user by calling setHeader:

self.client.setHeader('Content-Type', 'text/csv')

This example indicates that the value sent back to the user is actually comma-separated value content (eg. something to be loaded into a spreadsheet or database).

Adding a new field to the classic schema

This example shows how to add a simple field (a due date) to the default classic schema. It does not add any additional behaviour, such as enforcing the due date, or causing automatic actions to fire if the due date passes.

You add new fields by editing the schema.py file in you tracker’s home. Schema changes are automatically applied to the database on the next tracker access (note that roundup-server would need to be restarted as it caches the schema).

1. Modify the schema.py:

   issue = IssueClass(db, "issue",
                   assignedto=Link("user"), keyword=Multilink("keyword"),
                   priority=Link("priority"), status=Link("status"),
                   due_date=Date())
   

2. Add an edit field to the issue.item.html template:

   <tr>
    <th>Due Date</th>
    <td tal:content="structure context/due_date/field" />
   </tr>

   If you want to show only the date part of due_date then do this instead:

   <tr>
    <th>Due Date</th>
    <td tal:content="structure python:context.due_date.field(format='%Y-%m-%d')" />
   </tr>

3. Add the property to the issue.index.html page:

   (in the heading row)
     <th tal:condition="request/show/due_date">Due Date</th>
   (in the data row)
     <td tal:condition="request/show/due_date"
         tal:content="i/due_date" />

   If you want format control of the display of the due date you can enter the following in the data row to show only the actual due date:

   <td tal:condition="request/show/due_date"
       tal:content="python:i.due_date.pretty('%Y-%m-%d')">&nbsp;</td>

4. Add the property to the issue.search.html page:

   <tr tal:define="name string:due_date">
     <th i18n:translate="">Due Date:</th>
     <td metal:use-macro="search_input"></td>
     <td metal:use-macro="column_input"></td>
     <td metal:use-macro="sort_input"></td>
     <td metal:use-macro="group_input"></td>
   </tr>

5. If you wish for the due date to appear in the standard views listed in the sidebar of the web interface then you’ll need to add “due_date” to the columns and columns_showall lists in your page.html:

   columns string:id,activity,due_date,title,creator,status;
   columns_showall string:id,activity,due_date,title,creator,assignedto,status;

Adding a new constrained field to the classic schema

This example shows how to add a new constrained property (i.e. a selection of distinct values) to your tracker.

# add any additional database schema configuration here

category = Class(db, "category", name=String())
category.setkey("name")

issue = IssueClass(db, "issue", ... ,
    category=Multilink("category"), ... )

# add any additional database creation steps here - but only if you
# haven't initialised the database with the admin "initialise" command

category = db.getclass('category')
category.create(name="scipy")
category.create(name="chaco")
category.create(name="weave")

% roundup-admin -i <tracker home>
Roundup <version> ready for input.
Type "help" for help.
roundup> create category name=scipy
1
roundup> create category name=chaco
2
roundup> create category name=weave
3
roundup> exit...
There are unsaved changes. Commit them (y/N)? y

# Assign the access and edit permissions for issue, file and message
# to regular users now
for cl in 'issue', 'file', 'msg', 'category':
    p = db.security.getPermission('View', cl)
    db.security.addPermissionToRole('User', 'View', cl)
    db.security.addPermissionToRole('User', 'Edit', cl)
    db.security.addPermissionToRole('User', 'Create', cl)

<p class="classblock"
   tal:condition="python:request.user.hasPermission('View', 'category')">
 <b>Categories</b><br>
 <a tal:condition="python:request.user.hasPermission('Edit', 'category')"
    href="category?@template=item">New Category<br></a>
</p>

<!-- category.item -->

<tal:block metal:use-macro="templates/page/macros/icing">
 <title metal:fill-slot="head_title">Category editing</title>
 <td class="page-header-top" metal:fill-slot="body_title">
  <h2>Category editing</h2>
 </td>
 <td class="content" metal:fill-slot="content">

<form method="POST" onSubmit="return submit_once()"
      enctype="multipart/form-data">

<input type="hidden" name="@required" value="name">

<table class="form">
 <tr><th class="header" colspan="2">Category</th></tr>

<tr>
 <th>Name</th>
 <td tal:content="structure python:context.name.field(size=60)">
 name</td>
</tr>

<tr>
 <td>&nbsp;</td>
 <td colspan="3" tal:content="structure context/submit">
  submit button will go here
 </td>
</tr>

 </td>
</tal:block>

<!-- category.item -->
<tal:block metal:use-macro="templates/page/macros/icing">
 <title metal:fill-slot="head_title">Category editing</title>
 <td class="page-header-top" metal:fill-slot="body_title">
  <h2>Category editing</h2>
 </td>
 <td class="content" metal:fill-slot="content">
  <form method="POST" onSubmit="return submit_once()"
        enctype="multipart/form-data">

   <table class="form">
    <tr><th class="header" colspan="2">Category</th></tr>

    <tr>
     <th>Name</th>
     <td tal:content="structure python:context.name.field(size=60)">
     name</td>
    </tr>

    <tr>
     <td>
       &nbsp;
       <input type="hidden" name="@required" value="name">
     </td>
     <td colspan="3" tal:content="structure context/submit">
      submit button will go here
     </td>
    </tr>
   </table>
  </form>
 </td>
</tal:block>

<th>Category</th>
<td>
 <span tal:replace="structure context/category/field" />
 <span tal:replace="structure python:db.category.classhelp('name',
             property='category', width='200')" />
</td>

<tr tal:define="name string:category;
                db_klass string:category;
                db_content string:name;">
  <th>Priority:</th>
  <td metal:use-macro="search_select"></td>
  <td metal:use-macro="column_input"></td>
  <td metal:use-macro="sort_input"></td>
  <td metal:use-macro="group_input"></td>
</tr>

<tr>
  <th>Category:</th>
  <td>
    <select name="category">
      <option value="">don't care</option>
      <option value="">------------</option>
      <option value="1">scipy</option>
      <option value="2">chaco</option>
      <option value="3">weave</option>
    </select>
  </td>
  <td><input type="checkbox" name=":columns" value="category"></td>
  <td><input type="radio" name=":sort0" value="category"></td>
  <td><input type="radio" name=":group0" value="category"></td>
</tr>

<th tal:condition="request/show/category">Category</th>

<td tal:condition="request/show/category"
    tal:content="i/category"></td>

Adding a time log to your issues

We want to log the dates and amount of time spent working on issues, and be able to give a summary of the total time spent on a particular issue.

1. Add a new class to your tracker schema.py:

   # storage for time logging
   timelog = Class(db, "timelog", period=Interval())
   

   Note that we automatically get the date of the time log entry creation through the standard property “creation”.

   You will need to grant “Creation” permission to the users who are allowed to add timelog entries. You may do this with:

   db.security.addPermissionToRole('User', 'Create', 'timelog')
   db.security.addPermissionToRole('User', 'View', 'timelog')
   

   If users are also able to edit timelog entries, then also include:

   db.security.addPermissionToRole('User', 'Edit', 'timelog')
   

2. Link to the new class from your issue class (again, in schema.py):

   issue = IssueClass(db, "issue",
                   assignedto=Link("user"), keyword=Multilink("keyword"),
                   priority=Link("priority"), status=Link("status"),
                   times=Multilink("timelog"))
   

   the “times” property is the new link to the “timelog” class.

3. We’ll need to let people add in times to the issue, so in the web interface we’ll have a new entry field. This is a special field because unlike the other fields in the issue.item template, it affects a different item (a timelog item) and not the template’s item (an issue). We have a special syntax for form fields that affect items other than the template default item (see the cgi documentation on special form variables). In particular, we add a field to capture a new timelog item’s period:

   <tr>
    <th>Time Log</th>
    <td colspan=3><input type="text" name="timelog-1@period" />
     (enter as '3y 1m 4d 2:40:02' or parts thereof)
    </td>
   </tr>

   and another hidden field that links that new timelog item (new because it’s marked as having id “-1”) to the issue item. It looks like this:

   <input type="hidden" name="@link@times" value="timelog-1" />

   On submission, the “-1” timelog item will be created and assigned a real item id. The “times” property of the issue will have the new id added to it.

   The full entry will now look like this:

   <tr>
    <th>Time Log</th>
    <td colspan=3><input type="text" name="timelog-1@period" />
     (enter as '3y 1m 4d 2:40:02' or parts thereof)
     <input type="hidden" name="@link@times" value="timelog-1" />
    </td>
   </tr>

4. We want to display a total of the timelog times that have been accumulated for an issue. To do this, we’ll need to actually write some Python code, since it’s beyond the scope of PageTemplates to perform such calculations. We do this by adding a module timespent.py to the extensions directory in our tracker. The contents of this file is as follows:

   from roundup import date
   
   def totalTimeSpent(times):
       ''' Call me with a list of timelog items (which have an
           Interval "period" property)
       '''
       total = date.Interval('0d')
       for time in times:
           total += time.period._value
       return total
   
   def init(instance):
       instance.registerUtil('totalTimeSpent', totalTimeSpent)
   

   We will now be able to access the totalTimeSpent function via the utils variable in our templates, as shown in the next step.

5. Display the timelog for an issue:

   <table class="otherinfo" tal:condition="context/times">
    <tr><th colspan="3" class="header">Time Log
     <tal:block
          tal:replace="python:utils.totalTimeSpent(context.times)" />
    </th></tr>
    <tr><th>Date</th><th>Period</th><th>Logged By</th></tr>
    <tr tal:repeat="time context/times">
     <td tal:content="time/creation"></td>
     <td tal:content="time/period"></td>
     <td tal:content="time/creator"></td>
    </tr>
   </table>

   I put this just above the Messages log in my issue display. Note our use of the totalTimeSpent method which will total up the times for the issue and return a new Interval. That will be automatically displayed in the template as text like “+ 1y 2:40” (1 year, 2 hours and 40 minutes).

6. If you’re using a persistent web server - roundup-server or mod_python for example - then you’ll need to restart that to pick up the code changes. When that’s done, you’ll be able to use the new time logging interface.

An extension of this modification attaches the timelog entries to any change message entered at the time of the timelog entry:

1. Add a link to the timelog to the msg class in schema.py:

   msg = FileClass(db, “msg”,

   author=Link(“user”, do_journal=’no’), recipients=Multilink(“user”, do_journal=’no’), date=Date(), summary=String(), files=Multilink(“file”), messageid=String(), inreplyto=String(), times=Multilink(“timelog”))

2. Add a new hidden field that links that new timelog item (new because it’s marked as having id “-1”) to the new message. The link is placed in issue.item.html in the same section that handles the timelog entry.

   It looks like this after this addition:

   <tr>
    <th>Time Log</th>
    <td colspan=3><input type="text" name="timelog-1@period" />
     (enter as '3y 1m 4d 2:40:02' or parts thereof)
     <input type="hidden" name="@link@times" value="timelog-1" />
     <input type="hidden" name="msg-1@link@times" value="timelog-1" />
    </td>
   </tr>

   The “times” property of the message will have the new id added to it.

3. Add the timelog listing from step 5. to the msg.item.html template so that the timelog entry appears on the message view page. Note that the call to totalTimeSpent is not used here since there will only be one single timelog entry for each message.

   I placed it after the Date entry like this:

   <tr>
    <th i18n:translate="">Date:</th>
    <td tal:content="context/date"></td>
   </tr>
   </table>
   
   <table class="otherinfo" tal:condition="context/times">
    <tr><th colspan="3" class="header">Time Log</th></tr>
    <tr><th>Date</th><th>Period</th><th>Logged By</th></tr>
    <tr tal:repeat="time context/times">
     <td tal:content="time/creation"></td>
     <td tal:content="time/period"></td>
     <td tal:content="time/creator"></td>
    </tr>
   </table>
   
   <table class="messages">

Tracking different types of issues

Sometimes you will want to track different types of issues - developer, customer support, systems, sales leads, etc. A single Roundup tracker is able to support multiple types of issues. This example demonstrates adding a system support issue class to a tracker.

1. Figure out what information you’re going to want to capture. OK, so this is obvious, but sometimes it’s better to actually sit down for a while and think about the schema you’re going to implement.

2. Add the new issue class to your tracker’s schema.py. Just after the “issue” class definition, add:

   # list our systems
   system = Class(db, "system", name=String(), order=Number())
   system.setkey("name")
   
   # store issues related to those systems
   support = IssueClass(db, "support",
                   assignedto=Link("user"), keyword=Multilink("keyword"),
                   status=Link("status"), deadline=Date(),
                   affects=Multilink("system"))
   

3. Copy the existing issue.* (item, search and index) templates in the tracker’s html to support.*. Edit them so they use the properties defined in the support class. Be sure to check for hidden form variables like “required” to make sure they have the correct set of required properties.

4. Edit the modules in the detectors, adding lines to their init functions where appropriate. Look for audit and react registrations on the issue class, and duplicate them for support.

5. Create a new sidebar box for the new support class. Duplicate the existing issues one, changing the issue class name to support.

6. Re-start your tracker and start using the new support class.

Optionally, you might want to restrict the users able to access this new class to just the users with a new “SysAdmin” Role. To do this, we add some security declarations:

db.security.addPermissionToRole('SysAdmin', 'View', 'support')
db.security.addPermissionToRole('SysAdmin', 'Create', 'support')
db.security.addPermissionToRole('SysAdmin', 'Edit', 'support')

You would then (as an “admin” user) edit the details of the appropriate users, and add “SysAdmin” to their Roles list.

Alternatively, you might want to change the Edit/View permissions granted for the issue class so that it’s only available to users with the “System” or “Developer” Role, and then the new class you’re adding is available to all with the “User” Role.

Using an external password validation source

We have a centrally-managed password changing system for our users. This results in a UN*X passwd-style file that we use for verification of users. Entries in the file consist of name:password where the password is encrypted using the standard UN*X crypt() function (see the crypt module in your Python distribution). An example entry would be:

admin:aamrgyQfDFSHw

Each user of Roundup must still have their information stored in the Roundup database - we just use the passwd file to check their password. To do this, we need to override the standard verifyPassword method defined in roundup.cgi.actions.LoginAction and register the new class. The following is added as externalpassword.py in the tracker extensions directory:

import os, crypt
from roundup.cgi.actions import LoginAction

class ExternalPasswordLoginAction(LoginAction):
    def verifyPassword(self, userid, password):
        '''Look through the file, line by line, looking for a
        name that matches.
        '''
        # get the user's username
        username = self.db.user.get(userid, 'username')

        # the passwords are stored in the "passwd.txt" file in the
        # tracker home
        file = os.path.join(self.db.config.TRACKER_HOME, 'passwd.txt')

        # see if we can find a match
        for ent in [line.strip().split(':') for line in
                                            open(file).readlines()]:
            if ent[0] == username:
                return crypt.crypt(password, ent[1][:2]) == ent[1]

        # user doesn't exist in the file
        return 0

def init(instance):
    instance.registerAction('login', ExternalPasswordLoginAction)

You should also remove the redundant password fields from the user.item template.

Using a UN*X passwd file as the user database

On some systems the primary store of users is the UN*X passwd file. It holds information on users such as their username, real name, password and primary user group.

Roundup can use this store as its primary source of user information, but it needs additional information too - email address(es), roundup Roles, vacation flags, roundup hyperdb item ids, etc. Also, “retired” users must still exist in the user database, unlike some passwd files in which the users are removed when they no longer have access to a system.

To make use of the passwd file, we therefore synchronise between the two user stores. We also use the passwd file to validate the user logins, as described in the previous example, using an external password validation source. We keep the user lists in sync using a fairly simple script that runs once a day, or several times an hour if more immediate access is needed. In short, it:

1. parses the passwd file, finding usernames, passwords and real names,
2. compares that list to the current roundup user list:
   1. entries no longer in the passwd file are retired
   2. entries with mismatching real names are updated
   3. entries only exist in the passwd file are created
3. send an email to administrators to let them know what’s been done.

The retiring and updating are simple operations, requiring only a call to retire() or set(). The creation operation requires more information though - the user’s email address and their Roundup Roles. We’re going to assume that the user’s email address is the same as their login name, so we just append the domain name to that. The Roles are determined using the passwd group identifier - mapping their UN*X group to an appropriate set of Roles.

The script to perform all this, broken up into its main components, is as follows. Firstly, we import the necessary modules and open the tracker we’re to work on:

import sys, os, smtplib
from roundup import instance, date

# open the tracker
tracker_home = sys.argv[1]
tracker = instance.open(tracker_home)

Next we read in the passwd file from the tracker home:

# read in the users from the "passwd.txt" file
file = os.path.join(tracker_home, 'passwd.txt')
users = [x.strip().split(':') for x in open(file).readlines()]

Handle special users (those to ignore in the file, and those who don’t appear in the file):

# users to not keep ever, pre-load with the users I know aren't
# "real" users
ignore = ['ekmmon', 'bfast', 'csrmail']

# users to keep - pre-load with the roundup-specific users
keep = ['comment_pool', 'network_pool', 'admin', 'dev-team',
        'cs_pool', 'anonymous', 'system_pool', 'automated']

Now we map the UN*X group numbers to the Roles that users should have:

roles = {
 '501': 'User,Tech',  # tech
 '502': 'User',       # finance
 '503': 'User,CSR',   # customer service reps
 '504': 'User',       # sales
 '505': 'User',       # marketing
}

Now we do all the work. Note that the body of the script (where we have the tracker database open) is wrapped in a try / finally clause, so that we always close the database cleanly when we’re finished. So, we now do all the work:

# open the database
db = tracker.open('admin')
try:
    # store away messages to send to the tracker admins
    msg = []

    # loop over the users list read in from the passwd file
    for user,passw,uid,gid,real,home,shell in users:
        if user in ignore:
            # this user shouldn't appear in our tracker
            continue
        keep.append(user)
        try:
            # see if the user exists in the tracker
            uid = db.user.lookup(user)

            # yes, they do - now check the real name for correctness
            if real != db.user.get(uid, 'realname'):
                db.user.set(uid, realname=real)
                msg.append('FIX %s - %s'%(user, real))
        except KeyError:
            # nope, the user doesn't exist
            db.user.create(username=user, realname=real,
                address='%s@ekit-inc.com'%user, roles=roles[gid])
            msg.append('ADD %s - %s (%s)'%(user, real, roles[gid]))

    # now check that all the users in the tracker are also in our
    # "keep" list - retire those who aren't
    for uid in db.user.list():
        user = db.user.get(uid, 'username')
        if user not in keep:
            db.user.retire(uid)
            msg.append('RET %s'%user)

    # if we did work, then send email to the tracker admins
    if msg:
        # create the email
        msg = '''Subject: %s user database maintenance

        %s
        '''%(db.config.TRACKER_NAME, '\n'.join(msg))

        # send the email
        smtp = smtplib.SMTP(db.config.MAILHOST)
        addr = db.config.ADMIN_EMAIL
        smtp.sendmail(addr, addr, msg)

    # now we're done - commit the changes
    db.commit()
finally:
    # always close the database cleanly
    db.close()

And that’s it!

Using an LDAP database for user information

A script that reads users from an LDAP store using http://python-ldap.sf.net/ and then compares the list to the users in the roundup user database would be pretty easy to write. You’d then have it run once an hour / day (or on demand if you can work that into your LDAP store workflow). See the example Using a UN*X passwd file as the user database for more information about doing this.

To authenticate off the LDAP store (rather than using the passwords in the Roundup user database) you’d use the same python-ldap module inside an extension to the cgi interface. You’d do this by overriding the method called verifyPassword on the LoginAction class in your tracker’s extensions directory (see using an external password validation source). The method is implemented by default as:

def verifyPassword(self, userid, password):
    ''' Verify the password that the user has supplied
    '''
    stored = self.db.user.get(self.userid, 'password')
    if password == stored:
        return 1
    if not password and not stored:
        return 1
    return 0

So you could reimplement this as something like:

def verifyPassword(self, userid, password):
    ''' Verify the password that the user has supplied
    '''
    # look up some unique LDAP information about the user
    username = self.db.user.get(self.userid, 'username')
    # now verify the password supplied against the LDAP store

Preventing SPAM

The following detector code may be installed in your tracker’s detectors directory. It will block any messages being created that have HTML attachments (a very common vector for spam and phishing) and any messages that have more than 2 HTTP URLs in them. Just copy the following into detectors/anti_spam.py in your tracker:

from roundup.exceptions import Reject

def reject_html(db, cl, nodeid, newvalues):
    if newvalues['type'] == 'text/html':
    raise Reject, 'not allowed'

def reject_manylinks(db, cl, nodeid, newvalues):
    content = newvalues['content']
    if content.count('http://') > 2:
    raise Reject, 'not allowed'

def init(db):
    db.file.audit('create', reject_html)
    db.msg.audit('create', reject_manylinks)

You may also wish to block image attachments if your tracker does not need that ability:

if newvalues['type'].startswith('image/'):
    raise Reject, 'not allowed'

Stop “nosy” messages going to people on vacation

When users go on vacation and set up vacation email bouncing, you’ll start to see a lot of messages come back through Roundup “Fred is on vacation”. Not very useful, and relatively easy to stop.

1. add a “vacation” flag to your users:

   user = Class(db, "user",
              username=String(),   password=Password(),
              address=String(),    realname=String(),
              phone=String(),      organisation=String(),
              alternate_addresses=String(),
              roles=String(), queries=Multilink("query"),
              vacation=Boolean())
   

2. So that users may edit the vacation flags, add something like the following to your user.item template:

   <tr>
    <th>On Vacation</th>
    <td tal:content="structure context/vacation/field">vacation</td>
   </tr>

3. edit your detector nosyreactor.py so that the nosyreaction() consists of:

   def nosyreaction(db, cl, nodeid, oldvalues):
       users = db.user
       messages = db.msg
       # send a copy of all new messages to the nosy list
       for msgid in determineNewMessages(cl, nodeid, oldvalues):
           try:
               # figure the recipient ids
               sendto = []
               seen_message = {}
               recipients = messages.get(msgid, 'recipients')
               for recipid in messages.get(msgid, 'recipients'):
                   seen_message[recipid] = 1
   
               # figure the author's id, and indicate they've received
               # the message
               authid = messages.get(msgid, 'author')
   
               # possibly send the message to the author, as long as
               # they aren't anonymous
               if (db.config.MESSAGES_TO_AUTHOR == 'yes' and
                       users.get(authid, 'username') != 'anonymous'):
                   sendto.append(authid)
               seen_message[authid] = 1
   
               # now figure the nosy people who weren't recipients
               nosy = cl.get(nodeid, 'nosy')
               for nosyid in nosy:
                   # Don't send nosy mail to the anonymous user (that
                   # user shouldn't appear in the nosy list, but just
                   # in case they do...)
                   if users.get(nosyid, 'username') == 'anonymous':
                       continue
                   # make sure they haven't seen the message already
                   if not seen_message.has_key(nosyid):
                       # send it to them
                       sendto.append(nosyid)
                       recipients.append(nosyid)
   
               # generate a change note
               if oldvalues:
                   note = cl.generateChangeNote(nodeid, oldvalues)
               else:
                   note = cl.generateCreateNote(nodeid)
   
               # we have new recipients
               if sendto:
                   # filter out the people on vacation
                   sendto = [i for i in sendto
                             if not users.get(i, 'vacation', 0)]
   
                   # map userids to addresses
                   sendto = [users.get(i, 'address') for i in sendto]
   
                   # update the message's recipients list
                   messages.set(msgid, recipients=recipients)
   
                   # send the message
                   cl.send_message(nodeid, msgid, note, sendto)
           except roundupdb.MessageSendError, message:
               raise roundupdb.DetectorError, message
   

   Note that this is the standard nosy reaction code, with the small addition of:

   # filter out the people on vacation
   sendto = [i for i in sendto if not users.get(i, 'vacation', 0)]
   

   which filters out the users that have the vacation flag set to true.

Adding in state transition control

Sometimes tracker admins want to control the states to which users may move issues. You can do this by following these steps:

1. make “status” a required variable. This is achieved by adding the following to the top of the form in the issue.item.html template:

   <input type="hidden" name="@required" value="status">

   This will force users to select a status.

2. add a Multilink property to the status class:

   stat = Class(db, "status", ... , transitions=Multilink('status'),
                ...)

   and then edit the statuses already created, either:

   1. through the web using the class list -> status class editor, or
   2. using the roundup-admin “set” command.

3. add an auditor module checktransition.py in your tracker’s detectors directory, for example:

   def checktransition(db, cl, nodeid, newvalues):
       ''' Check that the desired transition is valid for the "status"
           property.
       '''
       if not newvalues.has_key('status'):
           return
       current = cl.get(nodeid, 'status')
       new = newvalues['status']
       if new == current:
           return
       ok = db.status.get(current, 'transitions')
       if new not in ok:
           raise ValueError, 'Status not allowed to move from "%s" to "%s"'%(
               db.status.get(current, 'name'), db.status.get(new, 'name'))
   
   def init(db):
       db.issue.audit('set', checktransition)
   

4. in the issue.item.html template, change the status editing bit from:

   <th>Status</th>
   <td tal:content="structure context/status/menu">status</td>

   to:

   <th>Status</th>
   <td>
    <select tal:condition="context/id" name="status">
     <tal:block tal:define="ok context/status/transitions"
                tal:repeat="state db/status/list">
      <option tal:condition="python:state.id in ok"
              tal:attributes="
                   value state/id;
                   selected python:state.id == context.status.id"
              tal:content="state/name"></option>
     </tal:block>
    </select>
    <tal:block tal:condition="not:context/id"
               tal:replace="structure context/status/menu" />
   </td>

   which displays only the allowed status to transition to.

Blocking issues that depend on other issues

We needed the ability to mark certain issues as “blockers” - that is, they can’t be resolved until another issue (the blocker) they rely on is resolved. To achieve this:

1. Create a new property on the issue class: blockers=Multilink("issue"). To do this, edit the definition of this class in your tracker’s schema.py file. Change this:

   issue = IssueClass(db, "issue",
                   assignedto=Link("user"), keyword=Multilink("keyword"),
                   priority=Link("priority"), status=Link("status"))
   

   to this, adding the blockers entry:

   issue = IssueClass(db, "issue",
                   blockers=Multilink("issue"),
                   assignedto=Link("user"), keyword=Multilink("keyword"),
                   priority=Link("priority"), status=Link("status"))
   

2. Add the new blockers property to the issue.item.html edit page, using something like:

   <th>Waiting On</th>
   <td>
    <span tal:replace="structure python:context.blockers.field(showid=1,
                                 size=20)" />
    <span tal:replace="structure python:db.issue.classhelp('id,title',
                                 property='blockers')" />
    <span tal:condition="context/blockers"
          tal:repeat="blk context/blockers">
     <br>View: <a tal:attributes="href string:issue${blk/id}"
                  tal:content="blk/id"></a>
    </span>
   </td>

   You’ll need to fiddle with your item page layout to find an appropriate place to put it - I’ll leave that fun part up to you. Just make sure it appears in the first table, possibly somewhere near the “superseders” field.

3. Create a new detector module (see below) which enforces the rules:

   • issues may not be resolved if they have blockers
   • when a blocker is resolved, it’s removed from issues it blocks

   The contents of the detector should be something like this:

   def blockresolution(db, cl, nodeid, newvalues):
       ''' If the issue has blockers, don't allow it to be resolved.
       '''
       if nodeid is None:
           blockers = []
       else:
           blockers = cl.get(nodeid, 'blockers')
       blockers = newvalues.get('blockers', blockers)
   
       # don't do anything if there's no blockers or the status hasn't
       # changed
       if not blockers or not newvalues.has_key('status'):
           return
   
       # get the resolved state ID
       resolved_id = db.status.lookup('resolved')
   
       # format the info
       u = db.config.TRACKER_WEB
       s = ', '.join(['<a href="%sissue%s">%s</a>'%(
                       u,id,id) for id in blockers])
       if len(blockers) == 1:
           s = 'issue %s is'%s
       else:
           s = 'issues %s are'%s
   
       # ok, see if we're trying to resolve
       if newvalues['status'] == resolved_id:
           raise ValueError, "This issue can't be resolved until %s resolved."%s
   
   
   def resolveblockers(db, cl, nodeid, oldvalues):
       ''' When we resolve an issue that's a blocker, remove it from the
           blockers list of the issue(s) it blocks.
       '''
       newstatus = cl.get(nodeid,'status')
   
       # no change?
       if oldvalues.get('status', None) == newstatus:
           return
   
       resolved_id = db.status.lookup('resolved')
   
       # interesting?
       if newstatus != resolved_id:
           return
   
       # yes - find all the blocked issues, if any, and remove me from
       # their blockers list
       issues = cl.find(blockers=nodeid)
       for issueid in issues:
           blockers = cl.get(issueid, 'blockers')
           if nodeid in blockers:
               blockers.remove(nodeid)
               cl.set(issueid, blockers=blockers)
   
   def init(db):
       # might, in an obscure situation, happen in a create
       db.issue.audit('create', blockresolution)
       db.issue.audit('set', blockresolution)
   
       # can only happen on a set
       db.issue.react('set', resolveblockers)
   

   Put the above code in a file called “blockers.py” in your tracker’s “detectors” directory.

4. Finally, and this is an optional step, modify the tracker web page URLs so they filter out issues with any blockers. You do this by adding an additional filter on “blockers” for the value “-1”. For example, the existing “Show All” link in the “page” template (in the tracker’s “html” directory) looks like this:

   <a href="#"
      tal:attributes="href python:request.indexargs_url('issue', {
     '@sort': '-activity',
     '@group': 'priority',
     '@filter': 'status',
     '@columns': columns_showall,
     '@search_text': '',
     'status': status_notresolved,
     '@dispname': i18n.gettext('Show All'),
    })"
      i18n:translate="">Show All</a><br>

   modify it to add the “blockers” info to the URL (note, both the “@filter” and “blockers” values must be specified):

   <a href="#"
      tal:attributes="href python:request.indexargs_url('issue', {
     '@sort': '-activity',
     '@group': 'priority',
     '@filter': 'status,blockers',
     '@columns': columns_showall,
     '@search_text': '',
     'status': status_notresolved,
     'blockers': '-1',
     '@dispname': i18n.gettext('Show All'),
    })"
      i18n:translate="">Show All</a><br>

   The above examples are line-wrapped on the trailing & and should be unwrapped.

That’s it. You should now be able to set blockers on your issues. Note that if you want to know whether an issue has any other issues dependent on it (i.e. it’s in their blockers list) you can look at the journal history at the bottom of the issue page - look for a “link” event to another issue’s “blockers” property.

Add users to the nosy list based on the keyword

Let’s say we need the ability to automatically add users to the nosy list based on the occurance of a keyword. Every user should be allowed to edit their own list of keywords for which they want to be added to the nosy list.

Below, we’ll show that this change can be done with minimal understanding of the Roundup system, using only copy and paste.

This requires three changes to the tracker: a change in the database to allow per-user recording of the lists of keywords for which he wants to be put on the nosy list, a change in the user view allowing them to edit this list of keywords, and addition of an auditor which updates the nosy list when a keyword is set.

user = Class(db, "user",
                username=String(),   password=Password(),
                address=String(),    realname=String(),
                phone=String(),      organisation=String(),
                alternate_addresses=String(),
                queries=Multilink('query'), roles=String(),
                timezone=String(),
                nosy_keywords=Multilink('keyword'))

<tr>
 <th>Nosy Keywords</th>
 <td>
 <span tal:replace="structure context/nosy_keywords/field" />
 <span tal:replace="structure python:db.keyword.classhelp(property='nosy_keywords')" />
 </td>
</tr>

def init(db):
    db.issue.audit('create', update_kw_nosy)
    db.issue.audit('set', update_kw_nosy)

def update_kw_nosy(db, cl, nodeid, newvalues):
    '''Update the nosy list for changes to the keywords
    '''
    # nodeid will be None if this is a new node
    current = {}
    if nodeid is None:
        ok = ('new', 'yes')
    else:
        ok = ('yes',)
        # old node, get the current values from the node if they haven't
        # changed
        if not newvalues.has_key('nosy'):
            nosy = cl.get(nodeid, 'nosy')
            for value in nosy:
                if not current.has_key(value):
                    current[value] = 1

    # if the nosy list changed in this transaction, init from the new value
    if newvalues.has_key('nosy'):
        nosy = newvalues.get('nosy', [])
        for value in nosy:
            if not db.hasnode('user', value):
                continue
            if not current.has_key(value):
                current[value] = 1

    # add users with keyword in nosy_keywords to the nosy list
    if newvalues.has_key('keyword') and newvalues['keyword'] is not None:
        keyword_ids = newvalues['keyword']
        for keyword in keyword_ids:
            # loop over all users,
            # and assign user to nosy when keyword in nosy_keywords
            for user_id in db.user.list():
                nosy_kw = db.user.get(user_id, "nosy_keywords")
                found = 0
                for kw in nosy_kw:
                    if kw == keyword:
                        found = 1
                if found:
                    current[user_id] = 1

    # that's it, save off the new nosy list
    newvalues['nosy'] = current.keys()

Restricting updates that arrive by email

Roundup supports multiple update methods:

1. command line
2. plain email
3. pgp signed email
4. web access

in some cases you may need to prevent changes to properties by some of these methods. For example you can set up issues that are viewable only by people on the nosy list. So you must prevent unauthenticated changes to the nosy list.

Since plain email can be easily forged, it does not provide sufficient authentication in this senario.

To prevent this we can add a detector that audits the source of the transaction and rejects the update if it changes the nosy list.

Create the detector (auditor) module and add it to the detectors directory of your tracker:

from roundup import roundupdb, hyperdb

from roundup.mailgw import Unauthorized

def restrict_nosy_changes(db, cl, nodeid, newvalues):
    '''Do not permit changes to nosy via email.'''

    if not (newvalues.has_key('nosy')):
        # the nosy field has not changed so no need to check.
        return

    if db.tx_Source in ['web', 'email-sig-openpgp', 'cli' ]:
        # if the source of the transaction is from an authenticated
        # source or a privileged process allow the transaction.
        # Other possible sources: 'email'
        return

    # otherwise raise an error
    raise Unauthorized, \
        'Changes to nosy property not allowed via %s for this issue.'%\
        tx_Source

def init(db):
   ''' Install restrict_nosy_changes to run after other auditors.

       Allow initial creation email to set nosy.
       So don't execute: db.issue.audit('create', requestedbyauditor)

       Set priority to 110 to run this auditor after other auditors
       that can cause nosy to change.
   '''
   db.issue.audit('set', restrict_nosy_changes, 110)

This detector (auditor) will prevent updates to the nosy field if it arrives by email. Since it runs after other auditors (due to the priority of 110), it will also prevent changes to the nosy field that are done by other auditors if triggered by an email.

Note that db.tx_Source was not present in roundup versions before 1.4.22, so you must be running a newer version to use this detector. Read the CHANGES.txt document in the roundup source code for further details on tx_Source.

Restricting the list of users that are assignable to a task

1. In your tracker’s schema.py, create a new Role, say “Developer”:

   db.security.addRole(name='Developer', description='A developer')
   

2. Just after that, create a new Permission, say “Fixer”, specific to “issue”:

   p = db.security.addPermission(name='Fixer', klass='issue',
       description='User is allowed to be assigned to fix issues')
   

3. Then assign the new Permission to your “Developer” Role:

   db.security.addPermissionToRole('Developer', p)
   

4. In the issue item edit page (html/issue.item.html in your tracker directory), use the new Permission in restricting the “assignedto” list:

   <select name="assignedto">
    <option value="-1">- no selection -</option>
    <tal:block tal:repeat="user db/user/list">
    <option tal:condition="python:user.hasPermission(
                               'Fixer', context._classname)"
            tal:attributes="
               value user/id;
               selected python:user.id == context.assignedto"
            tal:content="user/realname"></option>
    </tal:block>
   </select>

For extra security, you may wish to setup an auditor to enforce the Permission requirement (install this as assignedtoFixer.py in your tracker detectors directory):

def assignedtoMustBeFixer(db, cl, nodeid, newvalues):
    ''' Ensure the assignedto value in newvalues is used with the
        Fixer Permission
    '''
    if not newvalues.has_key('assignedto'):
        # don't care
        return

    # get the userid
    userid = newvalues['assignedto']
    if not db.security.hasPermission('Fixer', userid, cl.classname):
        raise ValueError, 'You do not have permission to edit %s'%cl.classname

def init(db):
    db.issue.audit('set', assignedtoMustBeFixer)
    db.issue.audit('create', assignedtoMustBeFixer)

So now, if an edit action attempts to set “assignedto” to a user that doesn’t have the “Fixer” Permission, the error will be raised.

Users may only edit their issues

In this case, users registering themselves are granted Provisional access, meaning they have access to edit the issues they submit, but not others. We create a new Role called “Provisional User” which is granted to newly-registered users, and has limited access. One of the Permissions they have is the new “Edit Own” on issues (regular users have “Edit”.)

First up, we create the new Role and Permission structure in schema.py:

#
# New users not approved by the admin
#
db.security.addRole(name='Provisional User',
    description='New user registered via web or email')

# These users need to be able to view and create issues but only edit
# and view their own
db.security.addPermissionToRole('Provisional User', 'Create', 'issue')
def own_issue(db, userid, itemid):
    '''Determine whether the userid matches the creator of the issue.'''
    return userid == db.issue.get(itemid, 'creator')
p = db.security.addPermission(name='Edit', klass='issue',
    check=own_issue, description='Can only edit own issues')
db.security.addPermissionToRole('Provisional User', p)
p = db.security.addPermission(name='View', klass='issue',
    check=own_issue, description='Can only view own issues')
db.security.addPermissionToRole('Provisional User', p)

# Assign the Permissions for issue-related classes
for cl in 'file', 'msg', 'query', 'keyword':
    db.security.addPermissionToRole('Provisional User', 'View', cl)
    db.security.addPermissionToRole('Provisional User', 'Edit', cl)
    db.security.addPermissionToRole('Provisional User', 'Create', cl)
for cl in 'priority', 'status':
    db.security.addPermissionToRole('Provisional User', 'View', cl)

# and give the new users access to the web and email interface
db.security.addPermissionToRole('Provisional User', 'Web Access')
db.security.addPermissionToRole('Provisional User', 'Email Access')

# make sure they can view & edit their own user record
def own_record(db, userid, itemid):
    '''Determine whether the userid matches the item being accessed.'''
    return userid == itemid
p = db.security.addPermission(name='View', klass='user', check=own_record,
    description="User is allowed to view their own user details")
db.security.addPermissionToRole('Provisional User', p)
p = db.security.addPermission(name='Edit', klass='user', check=own_record,
    description="User is allowed to edit their own user details")
db.security.addPermissionToRole('Provisional User', p)

Then, in config.ini, we change the Role assigned to newly-registered users, replacing the existing 'User' values:

[main]
...
new_web_user_roles = 'Provisional User'
new_email_user_roles = 'Provisional User'

All users may only view and edit issues, files and messages they create

Replace the standard “classic” tracker View and Edit Permission assignments for the “issue”, “file” and “msg” classes with the following:

def checker(klass):
    def check(db, userid, itemid, klass=klass):
        return db.getclass(klass).get(itemid, 'creator') == userid
    return check
for cl in 'issue', 'file', 'msg':
    p = db.security.addPermission(name='View', klass=cl,
        check=checker(cl))
    db.security.addPermissionToRole('User', p)
    p = db.security.addPermission(name='Edit', klass=cl,
        check=checker(cl))
    db.security.addPermissionToRole('User', p)
    db.security.addPermissionToRole('User', 'Create', cl)

Moderating user registration

You could set up new-user moderation in a public tracker by:

1. creating a new highly-restricted user role “Pending”,
2. set the config new_web_user_roles and/or new_email_user_roles to that role,
3. have an auditor that emails you when new users are created with that role using roundup.mailer
4. edit the role to “User” for valid users.

Some simple javascript might help in the last step. If you have high volume you could search for all currently-Pending users and do a bulk edit of all their roles at once (again probably with some simple javascript help).

Adding action links to the index page

Add a column to the item.index.html template.

Resolving the issue:

<a tal:attributes="href
   string:issue${i/id}?:status=resolved&:action=edit">resolve</a>

“Take” the issue:

<a tal:attributes="href
   string:issue${i/id}?:assignedto=${request/user/id}&:action=edit">take</a>

... and so on.

Colouring the rows in the issue index according to priority

A simple tal:attributes statement will do the bulk of the work here. In the issue.index.html template, add this to the <tr> that displays the rows of data:

<tr tal:attributes="class string:priority-${i/priority/plain}">

and then in your stylesheet (style.css) specify the colouring for the different priorities, as follows:

tr.priority-critical td {
    background-color: red;
}

tr.priority-urgent td {
    background-color: orange;
}

and so on, with far less offensive colours :)

Editing multiple items in an index view

To edit the status of all items in the item index view, edit the issue.item.html:

1. add a form around the listing table (separate from the existing index-page form), so at the top it reads:

   <form method="POST" tal:attributes="action request/classname">
    <table class="list">

   and at the bottom of that table:

    </table>
   </form

   making sure you match the </table> from the list table, not the navigation table or the subsequent form table.

2. in the display for the issue property, change:

   <td tal:condition="request/show/status"
       tal:content="python:i.status.plain() or default">&nbsp;</td>

   to:

   <td tal:condition="request/show/status"
       tal:content="structure i/status/field">&nbsp;</td>

   this will result in an edit field for the status property.

3. after the tal:block which lists the index items (marked by tal:repeat="i batch") add a new table row:

   <tr>
    <td tal:attributes="colspan python:len(request.columns)">
     <input type="submit" value=" Save Changes ">
     <input type="hidden" name="@action" value="edit">
     <tal:block replace="structure request/indexargs_form" />
    </td>
   </tr>

   which gives us a submit button, indicates that we are performing an edit on any changed statuses. The final tal:block will make sure that the current index view parameters (filtering, columns, etc) will be used in rendering the next page (the results of the editing).

Displaying only message summaries in the issue display

Alter the issue.item template section for messages to:

<table class="messages" tal:condition="context/messages">
 <tr><th colspan="5" class="header">Messages</th></tr>
 <tr tal:repeat="msg context/messages">
  <td><a tal:attributes="href string:msg${msg/id}"
         tal:content="string:msg${msg/id}"></a></td>
  <td tal:content="msg/author">author</td>
  <td class="date" tal:content="msg/date/pretty">date</td>
  <td tal:content="msg/summary">summary</td>
  <td>
   <a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">
   remove</a>
  </td>
 </tr>
</table>

Enabling display of either message summaries or the entire messages

This is pretty simple - all we need to do is copy the code from the example displaying only message summaries in the issue display into our template alongside the summary display, and then introduce a switch that shows either the one or the other. We’ll use a new form variable, @whole_messages to achieve this:

<table class="messages" tal:condition="context/messages">
 <tal:block tal:condition="not:request/form/@whole_messages/value | python:0">
  <tr><th colspan="3" class="header">Messages</th>
      <th colspan="2" class="header">
        <a href="?@whole_messages=yes">show entire messages</a>
      </th>
  </tr>
  <tr tal:repeat="msg context/messages">
   <td><a tal:attributes="href string:msg${msg/id}"
          tal:content="string:msg${msg/id}"></a></td>
   <td tal:content="msg/author">author</td>
   <td class="date" tal:content="msg/date/pretty">date</td>
   <td tal:content="msg/summary">summary</td>
   <td>
    <a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">remove</a>
   </td>
  </tr>
 </tal:block>

 <tal:block tal:condition="request/form/@whole_messages/value | python:0">
  <tr><th colspan="2" class="header">Messages</th>
      <th class="header">
        <a href="?@whole_messages=">show only summaries</a>
      </th>
  </tr>
  <tal:block tal:repeat="msg context/messages">
   <tr>
    <th tal:content="msg/author">author</th>
    <th class="date" tal:content="msg/date/pretty">date</th>
    <th style="text-align: right">
     (<a tal:attributes="href string:?@remove@messages=${msg/id}&@action=edit">remove</a>)
    </th>
   </tr>
   <tr><td colspan="3" tal:content="msg/content"></td></tr>
  </tal:block>
 </tal:block>
</table>

Setting up a “wizard” (or “druid”) for controlled adding of issues

1. Set up the page templates you wish to use for data input. My wizard is going to be a two-step process: first figuring out what category of issue the user is submitting, and then getting details specific to that category. The first page includes a table of help, explaining what the category names mean, and then the core of the form:

   <form method="POST" onSubmit="return submit_once()"
         enctype="multipart/form-data">
     <input type="hidden" name="@template" value="add_page1">
     <input type="hidden" name="@action" value="page1_submit">
   
     <strong>Category:</strong>
     <tal:block tal:replace="structure context/category/menu" />
     <input type="submit" value="Continue">
   </form>

   The next page has the usual issue entry information, with the addition of the following form fragments:

   <form method="POST" onSubmit="return submit_once()"
         enctype="multipart/form-data"
         tal:condition="context/is_edit_ok"
         tal:define="cat request/form/category/value">
   
     <input type="hidden" name="@template" value="add_page2">
     <input type="hidden" name="@required" value="title">
     <input type="hidden" name="category" tal:attributes="value cat">
      .
      .
      .
   </form>

   Note that later in the form, I use the value of “cat” to decide which form elements should be displayed. For example:

   <tal:block tal:condition="python:cat in '6 10 13 14 15 16 17'.split()">
    <tr>
     <th>Operating System</th>
     <td tal:content="structure context/os/field"></td>
    </tr>
    <tr>
     <th>Web Browser</th>
     <td tal:content="structure context/browser/field"></td>
    </tr>
   </tal:block>

   ... the above section will only be displayed if the category is one of 6, 10, 13, 14, 15, 16 or 17.

3. Determine what actions need to be taken between the pages - these are usually to validate user choices and determine what page is next. Now encode those actions in a new Action class (see defining new web actions):

   from roundup.cgi.actions import Action
   
   class Page1SubmitAction(Action):
       def handle(self):
           ''' Verify that the user has selected a category, and then move
               on to page 2.
           '''
           category = self.form['category'].value
           if category == '-1':
               self.client.error_message.append('You must select a category of report')
               return
           # everything's ok, move on to the next page
           self.client.template = 'add_page2'
   
   def init(instance):
       instance.registerAction('page1_submit', Page1SubmitAction)
   

4. Use the usual “new” action as the @action on the final page, and you’re done (the standard context/submit method can do this for you).