InvenioRDM Alpha 3 (December Release)

Projects Invenio RDM
1

Hohoho :santa: here is the December release!

Thank you for the feedback on the November release. We improved the documentation, pinned dependencies and made various other contributions that should help you because of it. Keep your feedback coming for this release too.

Release Content

In this release, we focused our efforts on:

  • Providing a default user interface for the instance.
    • The frontpage, the search page and the record page have been styled.
  • Uploading files to records
  • Previewing and downloading files from records
  • Updating some of the cli functionality

And we fixed various bugs as usual.

What do you need to do?

Try out the release and see if the above features work by following the updated
installation guide from the beginning (from scratch). Some of invenio-cli's options have changed.

Leave all feedback as comments to this thread using the template below.

## Bugs

## What worked well

## What didn't work well

## Wishes for documentation
Happy holidays from the InvenioRDM team!
2

And Happy holidays to you all too!

I am using Red Hat Enterprise Linux Server release 7.7 (Maipo) I gave a try and this is so far what I get, it hung when trying to build the docker image.

Bugs

(alpha3) ~/alpha3/december-release$ invenio-cli build --pre --containers
Building RDM application...
Locking dependencies...
Bootstrapping server...
Building base docker image...

In the /var/log/message
I see these type of errors:

An error occurred while installing cchardet==2.1.5 --hash=sha256:0f7ec49fcd28088c387d4afcc02c0549434d9e07deb2519365a6baa5b6c7ebb4 --hash=sha256:1a6d00b7cbd8acfc5e3093cb5f983a667d0752dc328123c8dcb293e252bfb024 --hash=sha256:240efe3f255f916769458343840b9c6403cf3192720bc5129792cbcb88bf72fb --hash=sha256:30f461d876cf3ea40c6fd949b9725c7c6e2522a3e87d33817d221e9f478d7e4d --hash=sha256:379a0bbd630bca677990df7509672a2ca43faf928939fd4b063fc2215b025b91 --hash=sha256:3ae84e6ee215925cd06a772d87c17d5485d862e2f1677aa0d6c295ea9313f117 --hash=sha256:4001620ba761b2ddd51caef6194444b5cd2f131de7c8c51a0f4896cb1ea1111a --hash=sha256:4bd54ff3a239b4fe598ba262d8730372e339fdd314286ceb6706a003d3e03d7b --hash=sha256:4d015296e96c0b2022495e4685b6fc0f3c9feed88fb062135f7f4748df7e0921 --hash=sha256:5011ab33557913489c98d2fbdd7d88f06736f0bb456c60952fc5e52886b2a410 --hash=sha256:6a192cce3009c9cd671588574ad0cb81322c78265ebcb33b2def63c15e44ea47 --hash=sha256:6bf07931fa81238d9174266aaf83605204192977671ef230d5651a8f9d4acf56 --hash=sha256:7f22a8194c4e696cea3eff28723f77858495dec52baf93261943c8bb8ce08035 --hash=sha256:8126798ec34b9fb444472d849b6510817939347809b898a0d6d6463e41c5901a --hash=sha256:8e3a50bcad2ca0921fbbd46d29cc215dcc0d6d360570d594aeb7b0e2de716e8c --hash=sha256:92341348fed2fb53899e9cccf030da5377beb8ed26dfddc6acf87f1f0ce4b80e --hash=sha256:950fb40918772efe5779747a2f6c83a053a26b623a674f1d4f271b35331a9968 --hash=sha256:a4e346151042b5cfae34fff65911842f04849be4a74f22bc52b1e99c11650210 --hash=sha256:af48965b752490d8e330e41a46ba47f07c63f22ac5c7f4c396b7efd3958daa2e --hash=sha256:b5cebf47f498e5ad4a9a5ef089b7ab6ef7926eaeea0b239c8e54f8217ce81cf2 --hash=sha256:b7cad0a062675acb42eb5170b07be774a5d9ca35a24388e918e5b78cb40ccbf2 --hash=sha256:bb05580cd40f4cb7ccda5f90163fc43e27820046a6d0af11c1747d515fc69859 --hash=sha256:f87bdef26758a0a8de93bbfd7651ac4fcf798a7a06c049c347a0103279698b23! Will try again.
3

OK I gave a try using RHEL 7 workstation OS and using Docker CE docker -ce-18.09.0-3.el7.x86_64, docker -ce-cli-18.09.0-3.el7.x86_64

Bugs

If I install docker-compose within the virtualenv using PIP method I get a problem with the Invenio-cli

  Using cached https://files.pythonhosted.org/packages/27/15/2cd0a203f318c2240b42cd9dd13c931ddd61067809fee3479f44f086103e/PyNaCl-1.3.0-cp34-abi3-manylinux1_x86_64.whl
Requirement already satisfied: more-itertools in /opt/invenio/.venv/lib/python3.6/site-packages (from zipp>=0.5->importlib-metadata; python_version < "3.8"->jsonschema<4,>=2.5.1->docker-compose) (8.0.2)
Collecting cffi!=1.11.3,>=1.8
  Using cached https://files.pythonhosted.org/packages/49/72/0d42f94fe94afa8030350c26e9d787219f3f008ec9bf6b86c66532b29236/cffi-1.13.2-cp36-cp36m-manylinux1_x86_64.whl
Processing /opt/invenio/.cache/pip/wheels/f2/9a/90/de94f8556265ddc9d9c8b271b0f63e57b26fb1d67a45564511/pycparser-2.19-py2.py3-none-any.whl
ERROR: invenio-cli 1.0.0a8 has requirement PyYAML<5.2.0,>=5.1.2, but you'll have pyyaml 3.13 which is incompatible.

What worked well

This time I could complete the proposed documentation until the advance example. It is not clear to me how to create the user.

What didn’t work well

The advance part was not clear to me, while I understand the goal of demonstrating restriction of privileges it documentation does not provide specific details in terms of the superuser creation. Probably something I should do following invenio roles creation commands or the lack functionality if the Invenio-cli.

Wishes for documentation

Is not clear for me whether this documentation already exists account creation within the invenioRDM…

4

Hello @cfgamboa we will document the user creation/permission granting as soon as possible.

As for the hanging error. Did it download the latest cookiecutter? (it should if you typed “yes” when initializing). Alternatively, you can confirm this by folder modification date (Cookiecutters are usually located in the home directory under “.cookiecutters”, for example “~/.cookiecutters”); the folder we are looking for is called “cookiecutter-invenio-rdm”.

You mention the error was in “var/log/messages”. Was there anything in “~/alpha3/december-release/logs/invenio-cli.log”?

Thanks a lot!
Pablo

5

Hi @ppanero

Ok thank you.
The latest .cookiecutters

ls -lrt
total 24
-rw-rw-r-- 1 invenio invenio 1090 Dec 24 11:48 LICENSE
-rw-rw-r-- 1 invenio invenio  323 Dec 24 11:48 AUTHORS.rst
drwxrwxr-x 2 invenio invenio   96 Dec 24 11:48 scripts
-rwxrwxr-x 1 invenio invenio 1716 Dec 24 11:48 run-tests.sh
-rw-rw-r-- 1 invenio invenio 2330 Dec 24 11:48 README.rst
drwxrwxr-x 2 invenio invenio   32 Dec 24 11:48 hooks
-rw-rw-r-- 1 invenio invenio  566 Dec 24 11:48 cookiecutter.json
drwxrwxr-x 4 invenio invenio 4096 Dec 24 11:48 {{cookiecutter.project_shortname}}
[invenio@invtest08 cookiecutter-invenio-rdm]$

The error log provided in the /log/invenio-cli.log did not show any apparent error.
All the best,
Carlos

6

Bugs

no bugs :smiley:

What worked well?

the installation worked very smooth no errors and bugs.

What didn’t work well?

according the documentation everything worked for me, BUT some things that i wanted to do further i think is not yet implemented so, i won’t go there.

bellow is the screenshot of me creating a record and then adding files to it.
i know the pics doesn’t match the metadata at all :smile:
but the uploading and creating a record worked.

previewr
previewr.PNG1164×1022 128 KB

1 Like
7

Hi @cfgamboa,

Trying to reproduce this on Red Hat 7.X (but with older docker!), we didn’t get the error you had… we got a different one though! Python dependencies seemed to have installed fine. It’s trying to build the docker image that failed:

docker.errors.BuildError: Please provide a source image with `from` prior to commit

I made a ticket for this that I am addressing now: https://github.com/inveniosoftware/cookiecutter-invenio-rdm/issues/31 . For your issue, I would install docker-compose globally (at least not in same virtualenv as invenio-cli) and try again. If you get the same error as above it will be a “good” sign.

EDIT Jan 9
This issue has been fixed in invenio-cli’s master branch (widely available in next release). Thanks for alerting us to issues in RHEL in the first place. Note that the build step does take a few minutes.

8

Hello,

Apologies for the weird formatting - this is a feature of the editor in this forum which I could not defeat.

The instructions on GitHub are missing the mention of the Python version dependency. Once I started installation under Python 3.5 it failed since it doesn’t handle the Posix path classes. Once I decided to upgrade all the way to 3.8 it didn’t work either because of a problem with shims i.e. ‘_shims’ object has no attribute InstallRequirement.
I’m using Ubuntu 16.04LTS.
When I went with 3.6 installation completed. Adding the user to docker group was necessary to avoid the socket error. But building images still failed -

Building RDM application…
Locking dependencies…
Bootstrapping server…
Building application docker image…
Traceback (most recent call last):
File “/home/maxim/p36/Python-3.6.10/rdm-env/bin/invenio-cli”, line 11, in
sys.exit(cli())
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/click/core.py”, line 764, in call
return self.main(*args, **kwargs)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/flask/cli.py”, line 586, in main
return super(FlaskGroup, self).main(*args, **kwargs)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/click/core.py”, line 717, in main
rv = self.invoke(ctx)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/click/core.py”, line 1137, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/click/core.py”, line 956, in invoke
return ctx.invoke(self.callback, **ctx.params)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/click/core.py”, line 555, in invoke
return callback(*args, **kwargs)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/invenio_cli/cli.py”, line 165, in build
log_config=invenio_cli.log_config)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/invenio_cli/helpers/scripts.py”, line 91, in bootstrap
_boostrap_containers(docker_helper, project_shortname)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/invenio_cli/helpers/scripts.py”, line 80, in _boostrap_containers
tag=’{}:latest’.format(project_shortname)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/invenio_cli/helpers/docker_helper.py”, line 42, in build_image
dockerfile=dockerfile, tag=tag)
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/docker/models/images.py”, line 287, in build
raise BuildError(chunk[‘error’], result_stream)
docker.errors.BuildError: The command ‘/bin/bash -c invenio collect --verbose && invenio webpack create && invenio webpack install --unsafe && invenio webpack build’ returned a non-zero code: 1

This seems to be related from the first command in the chain which produces
File “/home/maxim/p36/Python-3.6.10/rdm-env/lib/python3.6/site-packages/flask_babelex/init.py”, line 23, in
from werkzeug import ImmutableDict
ImportError: cannot import name ‘ImmutableDict’

Indeed this can be imported from wekrzeug.datastructures but not like this.

Could you please advise?

Thank you,

Maxim

9

Hi Maxim,

Thanks for the feedback (no worries about the editor formatting :wink: ). You were not the only one facing these issues. You will be happy to know that the upcoming February release will address these problems:

  • We will document the system dependencies better (in a subsequent release we will expand the supported Python versions). Python 3.6 is indeed the only version supported right now.
  • ImportError: cannot import name ‘ImmutableDict is something plaguing all modules since Werkzeug 1.0.0. The next release limits the version of Werkzeug, until upstream dependencies fix their imports too. In this particular case, the fix was merged in Flask-BabelEx 0.9.4 . If you limit Werkzeug: 'Werkzeug>=0.15.0,<1.0.0' in your dependencies, you should be fine.

Be on the look-out for this release at the beginning of March.