Set a correct backup status of ERROR when we fail to copy a file to an external directory given the backup configs demand it.
Also a lot of logs are added around this area to let admins easily find the problem if backup fails because of directory permissions.
Note we can only fix certain generic links, we can't handle ones like
/grade/report/{reportname}/index.php, but fortunately, we were only
using the generic ones.
I added a couple of other likely things too (badges and participants
list).
Files bigger than 4GB is not supported at this point. Zip archiver doesn't know about it and always returns true. We need to verify if it is a valid file or not and delete if not
Also delete the backup file immediately if anything goes wrong anywhere in backup. We don't want to use up space
Deleting directories at the end of backup can take a long time if
there are many directories (potentially up to 65536 file
directories). This change reports progress while deleting
directories to avoid timeout.
The new experimental setting enabletgzbackups allows backups to be
created so that the internal format for .mbz files is .tar.gz.
Restore transparently supports .mbz files with either internal
formats (.zip or .tar.gz).
The .tar.gz format has the following benefits for backup:
- Supports larger files (no limit on total size, 8GB on single file
vs. 4GB limit on total size)
- Compresses text better, resulting in smaller .mbz files.
- Reports progress regularly during compression of single files,
reducing the chance of timeouts during backups that include a
very large file.
Time performance may also be improved although I haven't done a
direct comparison.
Adds 'Preparing page display' progress bars for user interface pages
if they take a long time to display.
Also adds changes where other parts of the backup progress timed out on
long backups. After this change, and MDL-41838, it is finally possible on
my dev server to successfully back up the 'XL' test course.
1. Changes progress bar code to allow headings for progress bar (so users have
some clue what's going on if a page has more than one progress bar).
2. Changes restore code so that a progress bar can display during pre-checks if
they take longer than 5 seconds.
3. Changes pre-check and restore code so that, in various points where the system
can take a long time within an individual step, intederminate progress is
indicated and it won't time out.
When we import courses or duplicate activities, the deduplicating nature of
the file_storage system means that we don't actually have to include the
real files in the backup - just their metadata from the file table.
The restoration process needs to know not to expect files from the backup
phase so a flag is set in moodle_backup metadata.
This reduces the period of time that some of the unit tests
for calculate_next_automated_backup() are failing right now
(from 1week to just some hours around the DST changes).
Note it's not the final solution because they will still fail
when New_York or Brussels DST changes are about to happen, but
to get this completely fixed we'll need to wait to switch from
current custom TZ/DST support to PHP built-in one.
Ray Bon