Upgrade from v9 to v10

Hello!!
I want to upgrade from v9 to v10 using the documentation. But in the Re-index data section, the commands are not known by the system even so i am in “my-site” directory.
can anybody help ?
Thanks in advance

Hello,
do you have the latest version of invenio-cli?

$ pip show invenio-cli
Name: invenio-cli
Version: 1.0.14

I have the version 1.0.4

You will need to upgrade, see the step Upgrade your invenio-cli, the first step in the upgrade guide.

I did that, but i still have the version 1.0.4

I am not sure what is the command that you have used, I suggest the following:

$ pip uninstall -y invenio-cli
$ pip install invenio-cli

thanks, it’s working.

I got an error when i run pipenv run invenio alembic upgrade

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/bin/invenio", line 8, in <module>
    sys.exit(cli())
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/flask/cli.py", line 357, in decorator
    return __ctx.invoke(f, *args, **kwargs)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/flask_alembic/cli/click.py", line 91, in upgrade
    base.upgrade(target)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/flask_alembic/cli/base.py", line 85, in upgrade
    get_alembic().upgrade(target)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/flask_alembic/extension.py", line 327, in upgrade
    self.run_migrations(do_upgrade)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/flask_alembic/extension.py", line 236, in run_migrations
    env.run_migrations(**kwargs)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/alembic/runtime/environment.py", line 865, in run_migrations
    self.get_context().run_migrations(**kw)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/alembic/runtime/migration.py", line 611, in run_migrations
    for step in self._migrations_fn(heads, self):
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/flask_alembic/extension.py", line 325, in do_upgrade
    return self.script_directory._upgrade_revs(target, revision)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/alembic/script/base.py", line 439, in _upgrade_revs
    return [
  File "/usr/lib/python3.8/contextlib.py", line 131, in __exit__
    self.gen.throw(type, value, traceback)
  File "/home/openmrs/.local/share/virtualenvs/invenio-YWY9aZyz/lib/python3.8/site-packages/alembic/script/base.py", line 267, in _catch_revision_errors
    raise util.CommandError(err.args[0]) from err
alembic.util.exc.CommandError: Requested revision 41157f1933d6 overlaps with other requested revisions eb9743315a9d

Hello!

Did you try to restore the database at some point?

Can you provide us with the current migrations installed in your system? We can have a look and take it from there. To do so, run the following commands:

List current heads:

invenio alembic heads

then access the database and list all the alembic versions:

psql -U <name_of_your_instance>

<instance_name>=# SELECT * FROM alembic_version;

lastly, paste both results here.

Thank you! :slight_smile:

invenio alembic heads

dbdbc1b19cf2 -> 35c1075e6360 (default) (effective head), Force naming convention.
2da9a03b0833 (2e97565eba72, 862037093962) -> 1ba76da94103 (default, invenio_records_files) (head), Create records files tables.
676dd587542d -> e1146238edd3 (invenio_vocabularies) (head), Create awards table.
862037093962 -> 07fb52561c5c (invenio_records) (head), Alter column from json to jsonb.
bff1f190b9bd (62efc52773d4) -> aaa265b0afa6 (default, invenio_oauthclient) (head), Move UserIdentity to accounts.
dbe499bfda43 -> 04480be1593e (invenio_access) (head), Add ActionSystemRoles.
5b478fe7ef7f -> a3f5a8635cbb (invenio_communities) (head), Remove version table.
12a88921ada2 -> 4e57407b8e4a (default, invenio_oauth2server) (head), Add on delete cascade.
8ae99b034410 -> a29271fd78f8 (invenio_files_rest) (head), Create files object partial unique index.
e655021de0de -> 5d25c1981985 (invenio_oaiserver) (head), Add system_created field.
5cd30a3503c9 -> a14fa442680f (invenio_requests) (effective head), Create tables.
8ed1a438601c, 88d1463de5c0 (de9c14cbb0b2, a14fa442680f) -> 9e0ac518b9df (invenio_rdm_records) (head) (mergepoint), Create records/communities M2M table.
88d1463de5c0 -> a3957490361d (invenio_rdm_records) (head), Remove PIDRelations tables.
62efc52773d4 -> eb9743315a9d (invenio_accounts) (effective head), Add user profile and preferences as JSON fields to the User table.
f615cee99600 -> 999c62899c20 (invenio_pidstore) (head), Create pidstore tables.
c25ef2c50ffa (eb9743315a9d) -> 41157f1933d6 (invenio_userprofiles) (head), Remove table.

and

SELECT * FROM alembic_version

version_num
--------------
 1ba76da94103
 9e0ac518b9df
 a3957490361d
 4e57407b8e4a
 a29271fd78f8
 e655021de0de
 04480be1593e
 999c62899c20
 07fb52561c5c
 a3f5a8635cbb
 aaa265b0afa6
 eb9743315a9d
 41157f1933d6
 e1146238edd3

I fear your database migrations are currently in an inconsistent state. I have run the migration from a fresh instance and I got the following versions in the database:

1ba76da94103
9e0ac518b9df
a3957490361d
4e57407b8e4a
a29271fd78f8
04480be1593e
999c62899c20
07fb52561c5c
a3f5a8635cbb
aaa265b0afa6
41157f1933d6
e1146238edd3
5d25c1981985

As you can see, I do not have the following migrations:

e655021de0de
eb9743315a9d

And you do not have the following:

5d25c1981985

EDIT: it seems that the migration e655021de0de did not upgrade to 5d25c1981985. They are from invenio-oaiserver (see code) Thus, the difference.

Did you try to restore the database to an older version at some point? If so, it might have caused the issue.

Is your data safe to be wiped? If so, you can simply wipe it and run an upgrade again. To do so:

invenio db drop
invenio alembic upgrade

If the data is not meant to be wiped (e.g. production data), please let me know and we can try to fix the migrations using alembic solely.

It is a development environment. I followed your instructions and everything is fine now. Thanks you

1 Like