InvenioRDM Alpha 12 (October Release)

We’re happy to announce the InvenioRDM October release. Thank you to our team members for their efforts on this release.

In early October we held a virtual project workshop. As part of the workshop we have updated the roadmap. The roadmap is now publicly available in two versions, one simplified and one for project tracking. See below:

1- Simplified

2- GitHub for project tracking

What’s new?

Metadata Schema

The entire metadata schema for bibliographic records has been updated, and is now ready for a thorough review by all project partners. The update includes changes to the JSONSchema, Elasticsearch mappings, the REST API data validation layer as well as addition of many fields to the deposit form.

See https://inveniordm.docs.cern.ch/reference/metadata/ for a full reference of the new metadata schema.

User Experience

We have done a lot of work on improving the UX of the primary pages such as the frontpage, search results and record landing page. Below you can see some screenshots of before/after:

Search results

The search results have been tightened up as well. Also, we’ve added support for nested facets, so you can expand broader categories into subcategories (e.g., Publication and Image).

Landing page

The landing page you’ll notice now has Edit and New version buttons (only the Edit button currently works). The right column has been tightened up, and most of the new metadata fields are now properly displayed on the landing page.

Human readable labels

In the facets you’ll notice we now have human readable labels. Instead of ‘publication’ it will say ‘Publication’; instead of ‘open’ it will say ‘Open Access’.

Similarly to search results and landing pages, you’ll now see the correct resource type as well as icons on Open Access.

badges

Localization

Dates, like the publication date which supports Extended Date Time Format is now properly localized as well using the Unicode Common Locale Data Repository. For instance here an English and Turkish localization:

localization_english

localization_turkish

Affiliations

The affiliation display was redone:

affiliations

Manage section

There’s a new manage section on the record landing page, which allows you to edit a record.

manage

CLI Improvements

We have made it easier to get started with developing InvenioRDM. Partially to make our own developers’ life easier, but also to make it easier to customize InvenioRDM.

Shells

You can now easily activate the Python virtualenv shell as well start a Python terminal from your instance using the following commands:

invenio-cli shell
invenio-cli pyshell

Watching assets

We have now simplified how you can change styling via automatic watching of file changes on assets. Previously you had to manually rebuild the assets. Now instead you can simply execute the following commands, and the styling will automatically rebuild once the file changes:

invenio-cli assets --force --development
invenio-cli assets watch

Develop an Invenio module

Developers often need to install the latest development versions of Invenio modules to work on them. This can now easily be done with:

invenio-cli ext module-install ~/src/invenio-app-rdm ~/src/invenio-rdm-records
invenio-cli assets --force --development

Develop a React module

The above works only for Python packages. If instead, you are working on one of our React libraries, you can now easily install and watch the module for changes as well:

invenio-cli assets watch-module --link ~/src/react-invenio-deposit

Install

If you previously installed InvenioRDM, make sure you have the latest Docker image of your choice according to the Python version:

docker pull inveniosoftware/centos7-python:3.6
docker pull inveniosoftware/centos8-python:3.7
docker pull inveniosoftware/centos8-python:3.8

To install:

pip install --upgrade invenio-cli
invenio-cli init rdm
cd my-site
invenio-cli containerize --pre
invenio-cli demo --containers

To stop the instance without destroying the records that were created, you can just run:

cd my-site
invenio-cli stop

To destroy the Python virtualenv, and remove the docker containers, you can now just run:

cd my-site
invenio-cli destroy

Feedback

As always, we welcome your feedback. When you provide feedback on Discourse your message should be pre-populated with the classic template (bugs, what worked well, what didn’t work well, wishes for documentation).

Here is the template to give feedback if it’s not automated:


## Bugs

## What worked well

## What didn't work well

## Wishes for documentation

Take care and stay safe!

Bugs

nothing yet…

What worked well

invenio-cli init / invenio-cli install all smooth.
not sure when this popped up, but easy access to invenio.cfg via symlink is appreciated!

What didn’t work well

after invenio-cli install, needs effort to use non-dockerized services.

  • could cookiecutter ask if you want to use ‘external’ services, and collect the details for SQL, RabbitMQ, Redis, ES etc?
  • and/or commented examples in invenio.cfg
  • invenio-cli services should work in this situation too (ie give me an easy way to do db init create / queue init etc etc if I’m not using dockerized services)
  • same with demo records
    (I know these are possibly a bit specific to the way I’ve got things set up - but likely to be similar issues for anyone not using dockerized services in production?)

what is the point of APP_ALLOWED_HOSTS including 0.0.0.0 - I don’t think that will ever work (since Flask wants an exact IP match)?
probably need the -h from run to be put in here instead? That still doesn’t work if you bind to 0.0.0.0, but at least helps if you bind to a private IP (as I do when testing locally with a VM)

Wishes for documentation

document changes in invenio.cfg to use some or more external services
(seems inconsistent to have the DB URI but not other services)

specify that ‘description’ field is implicitly of type ‘abstract’

Bugs

none

What worked well

Straight forward installation
I like the new features shell and pyshell

Having done some more testing…

Bugs

The API will allow you to publish a totally empty record. This breaks the search UI (the whole page if it has any bad results on) as it tries to render a record with no metadata. Example below.

What worked well

Otherwise smoothly imported/published ~2000 (non-empty!) records as a test

What didn’t work well

“show affiliations” button (in record detail view) shouldn’t appear if no affiliations available

Would be nice if you could click on an author name (again in detail view, possibly elsewhere) to query by that author. Same for subject pills.

Wishes for documentation

steps to reproduce empty publishing bug.

On a fresh instance, submit an empty draft:

 curl -k -X POST https://10.10.127.100:5000/api/records -H "Content-Type: application/json" --data {} | jq '.'

{
  "metadata": {},
  "id": "7a9q5-xnx14",
  "created": "2020-11-20 10:11:28.261888",
  "expires_at": "2020-11-20 10:11:28.261923",
  "conceptid": "t74ef-6g956",
  "access": {},
  "revision_id": 1,
  "updated": "2020-11-20 10:11:28.271519",
  "links": {
    "publish": "https://10.10.127.100:5000/api/records/7a9q5-xnx14/draft/actions/publish",
    "self_html": "https://10.10.127.100:5000/uploads/7a9q5-xnx14",
    "self": "https://10.10.127.100:5000/api/records/7a9q5-xnx14/draft"
  }
} 

Attempt to publish it:

curl -k -X POST https://10.10.127.100:5000/api/records/7a9q5-xnx14/draft/actions/publish | jq '.'

{
  "metadata": {},
  "id": "7a9q5-xnx14",
  "created": "2020-11-20 10:11:46.907586",
  "conceptid": "t74ef-6g956",
  "access": {},
  "revision_id": 1,
  "updated": "2020-11-20 10:11:46.925311",
  "links": {
    "self_html": "https://10.10.127.100:5000/records/7a9q5-xnx14",
    "self": "https://10.10.127.100:5000/api/records/7a9q5-xnx14"
  }
}

Published record retrievable from /api/records/:

curl -k https://10.10.127.100:5000/api/records/7a9q5-xnx14 | jq '.'

{
  "metadata": {},
  "id": "7a9q5-xnx14",
  "created": "2020-11-20 10:11:46.907586",
  "conceptid": "t74ef-6g956",
  "access": {},
  "revision_id": 1,
  "updated": "2020-11-20 10:11:46.925311",
  "links": {
    "self_html": "https://10.10.127.100:5000/records/7a9q5-xnx14",
    "self": "https://10.10.127.100:5000/api/records/7a9q5-xnx14"
  }
}

One more - subject pills overlap if on >1 line, and should probably not break on whitespace where possible?

I got an error while trying to run an Invenio RDM instance:

ERROR: for invenio-rdm-demo_mq_1  Cannot start service mq: OCI runtime create failed: container_linux.go:349: starting container process caused "exec: \"docker-entrypoint.sh\": executable file not found in $PATH": unknown

Aftering investigating and looking up for quite some time, I found it was caused by the image used here i.e. rabbitmq:3-management. It is having a bug in it and the above error went away if I chose to use another tag e.g. rabbitmq:3.8-management

Thanks for the feedback @DanGranville, @cfgamboa and @astrowq! Issues were created in

1 Like