Historically it was possible, through a series of question restores,
moves and edits, to end up with multiple questions in the same
category with the same stamp, but differences in other question
or answer fields. This, combined with changes in versions, led
to errors when restoring or duplicating quizzes using these questions.
While recent changes have made it impossible to create this situation
in current Moodle versions, as any edits will create a new question
version with a new stamp, this situation may exist on long-standing
Moodle sites which have been upgraded since pre-4.0.
This change performs a much wider-ranging comparison of restored
existing questions, generating a hash of all the data for a question in
a backup file, and a corresponding hash for each question in the target
category, to decide if a restored question matches a question already in
the database.
This is mainly for the benefit of multilang users.
As well as changing the database schema, it is also necessary to change
where this is inforced in PHP code (forms, web services). This was missed
when shortname was changed from 100 to 255 chars some time ago, but I have
fixed that here.
Note, I have not changed the legacy mnet code, since as far as I am aware
that is deprecated.
Restore of categories and questions happens in several phases.
First, the file is scanned for which questions and categories
it contains, to work out if these are new questions which need
to be restored, or if they already exist in the database in a
place that can be used.
That code had not been updated to cope with the Moodle 4.0
versioning changes, so it is updated here (while still keeping
the code to cope with the old backup format.)
Prevent backups containing legacy MD5 hashed user passwords
from being added back into the database. Hashes are converted
to random SHA512 hashed passwords upon restore for these backups.
Several version checks were incorrectly using
restore_controller::info::moodle_release instead of moodle_version as a
version number. This replaces all of those checks with a common pair of
methods to make the checks clearer and more maintainable.
The versioning changes will require some major changes
in the backup and restore of question bank and its
elements. This change introduces those changes to make
it compatible with the new world of versioning in question
bank. This commit also removes quiz_slots fields and
quiz_slot_tags table.
This commit will also introduce the versioning db
structure and some major changes to the quiz
and quiz attempts for the question, random
question and the view.
This commit implements the behat changes for versioning
in core question and associated locations.
Though it's really rare for this to happen (it only was discovered
when running unit tests with Oracle), it's possible to get
problems restoring courses when context ids in the backup file
do match existing contexts in the restore target site in certain
ways (see the issue for more information).
This change just ensures that every call to the method (that
happens 4 times, for sys, coursecat, course and module levels):
prechek_precheck_qbanks_by_level()
Is processed always for that level. Before the patch it was
possible (hardly but possible) to return question categories
belonging to another level when some qcats contexts were matching
between the backup file and the target site.
1) Remove any floatval() casting. They are breaking / killing
.10 versions (converting them to .1). Since Moodle 2.0 all the
backup::RELEASE have been 100% numerical values.
2) Use version_compare() always to compare backup::RELEASE values.
They are always versions and the function is aware of versions
> .9, able to clean/ignore alpha chars... and everything else.
Note that I've also changed a couple of cases in formats (topics and
weeks) that were correct, but just added the same comment and used
the same version_compare() comparison parameters style, so all uses
in core are consistent (and safe to be copied out there).
Allow both UI and automated backups to be created without
including files. Instead include only file references.
This is essentially implementing "SAMESITE" to backup files
instead of only for import and export functionality.
A new backup setting to include files (defaults to yes)
has been included.
The restore process will also look for and attempt to
restore files from the trashdir as part of restoring
backups. Additionally to support this process the
ammount of time files are kept in trashdir before they
are cleaned up via cron is also adjustable via admin
setting.
They are really a pointer from a particular quiz to a question category.
They should never be shared, because if they are, unexpected things
happend when they are edited.
Any backup & restore operation may be leaving opened files
if a file logger is being used. This implementes the close()
method, so every logger can close any resource.
Also, the recommended backup_controlled::destroy() method
now calls to new logger::destroy() method in charge of
deleting all the references and closing any resource.
Finally, some internally used controllers, were missing
their destroy call, leading to associated loggers to
remain open. Now all them are explicitly deltroyed.
This commit contains three changes in the three files:
1. A bug in the backup process meant that anything logged after a
certain point did not appear in the on-screen display of the
backup log, because the logger was serialised and deserialised
but display code referred to the old version. Changed so that
code retrieves new object.
2. Add more information to backup log when there is a missing file.
3. Add more information to restore log when there is a missing file
(and remove existing code duplication of the current message).
The 'missing file' situation is one that generally shouldn't occur in
normal usage, but when it does happen, it is useful to have full
information about the file.
When restoring a backup that contains a large number of users
to a different server where those users don't already exist,
creating users can take a significant time. This change adds
progress reporting so that it doesn't time out and shows
activity in the user interface while doing the creation.
1. Changes progress bar code to allow headings for progress bar (so users have
some clue what's going on if a page has more than one progress bar).
2. Changes restore code so that a progress bar can display during pre-checks if
they take longer than 5 seconds.
3. Changes pre-check and restore code so that, in various points where the system
can take a long time within an individual step, intederminate progress is
indicated and it won't time out.
encode_backup_temp_info() and decode_backup_temp_info() have been
introduced to keep the info field encoding in one place.
Many locations used get_backup_ids_record() to obtain info, that
makes lots of calls to get_backup_ids_record() which can be slow.
We now complete all those inline by adding the info field to the query.
To reduce memory usage, all queries of that nature have been changed
to use get_recordset_*. gzcompress was introduced if available to minimize
traffic to/from the database and to decrease the memory required for caching.
The compression time is saved by the benefits in other places of having smaller data.
The cache is function local and testing against a large database
indicates 10k questions is a large category. Restore already
uses MEMORY_EXTRA and that will have enough space for the couple
of megabtyes a local sql hash will introduce.
Mark Johnson