This adds a few changes to the old test_context_not_used test:
- Move it to become an advanced_test, because its mission
is to verify that the assertEventContextNotUsed() assertion
works as expected.
- For consistency, also move the fixtures to own phpunit fixtures.
- Add proper coverage tags, to verify that the assertion is being
covered.
- Add a data provider to provide all the current cases and ease
any future case that may be needed in the future. One by one
because previously there was code never executed with the
warning expectation causing the test to stop.
- Run them in isolation, while this is not strictly required, it's
including external fixtures and, we'll need that isolation soon
(for changes coming when moving the test to PHPUnit 9.6 in MDL-81266).
The only tests that have COMPLETE* support for the debugging sink
are the advanced_testcase and the database_driver ones (store and
report). So we must ensure that the rest of tests don't use the
debugging sink at all.
Right now we are using it for storing, but later there is not
reporting, so any debugging happening within non advanced tests
is not detected.
This commit just ensures that we stop making that storing for
non advanced/database_driver tests. Nothing more, nothing less.
* Note that we have had to add a few missing bits to the
database_driver testcase because it was not 100% complete. Now
it behaves 100% the same than the advanced_testcase one regarding
the debugging sink.
- Added a new method called get_messages_by_component()
This method will help to retreive the redirected messages
of specific component only
- Added a new method called get_messages_by_component_and_type()
This method will help to retreive the redirected messages
of specific component and type only
During the bootstrap of PHPUnit we ensure that the database has been
reset to its initial state.
We do this by checking the internally-stored DB write count between
runs. If the count is not yet set (null), or it has been increased, we
force a reset.
When running an isolated test the test runner resets the database, it
then sets up a new isolated test environment by writing a new PHPUnit
test case and passing it to a new PHP Process using standard in. As part
of this, the bootstrap is run for that process.
Because we are in a new process, the db write count is fresh and not yet
set. This has been leading to an additional db reset before the isolated
test.
To handle this we want to _not_ perform a reset during the
initialisation for isolated runs. We know that the DB is in a fresh
state before we start the run.
To support this we need to know whether the test is an isolated test
during the bootstrap, which means we cannot use the previous approach to
calculating this.
Instead we look at the PHP_SELF value. PHP sets this to "Standard input
code" when run from stdin, instead of running a file.
There should not be any other legitimate reason to run a PHPUnit
bootstrap via this stdin approach.
Unfortunately this approach is a little bit risky as it depends on the
presence of a specific string, however this string has been in place
since 2016, and there is no legitimate way of calculating this.
I did consider looking at whether the called script included `/vendor/`
and `/phpunit`, but this is also likely a risky approach if someone
calls PHPUnit in an unexpected way.
This approach is itself unit tested so any change to PHP's stdin string
before we deprecate this approach entirely in 12 months time will be
caught.
When we deprecate the use of a file, we often include tests which ensure
that the legacy behaviour is maintained. There are also legacy uses
in the community where people would like to use the deprecated API for a
period.
The issue that we face is that, if the deprecated file is included once,
then it will be included for all other, unrelated, tests. This means
that other tests may not detect cases where the deprecated file was
included.
We can solve these cases by running the test that performs the inclusion
in a deprecated process. This means that the inclusion is only performed
in that isolated process, and other unrelated tests do not include the
file.
However, we also then need to detect which files which are including the
file and which we do not know about.
This change introduces:
- an override to the TestCase::setInIsolation method to define a
constant when the test is running in isolation
- a new function that a file can call when it is included, to make sure
that the test process was isolated, where there is any test.
The PHP_CodeSniffer @codingStandardsIgnore annotations are deprecated
and, since version 3.x, the new // phpcs:ignore comments should be used
instead.
This commits just reviews all the uses in core, replacing them for
the better new candidate, or removing when no longer needed.
Statically cache list of loaded reports during request lifecycle,
this ensures that computationally heavy initialisation routines
in system reports are only executed once (e.g. the access tab).
* The basic_test::assertTag method will issue a warning as $tag is not
found, failing the PHP Unit test that uses this method.
* Add tests to check that assertTag is working
Following the changes in the schema, from "whitelists" to "includes",
we are deprecating these two properties from phpunit_coverage_info:
- whitelistfolders => includelistfolders
- whitelistfiles => includelistfiles
They will continue working over the deprecation period but the init/util
scripts will throw some warnings about them being deprecated for 3.11 and
the way to replace them.
Standard 2y deprecation applies with final removed to happen @ MDL-71067
This performs all the changes needed in the util generator to
produce XML files compliant with the new schema (see previous
commit for description of changes).
- All the occurrences in code of filter => coverage.
- All the occurrences in code of whitelist => include.
- Apply all the changes to comply with the new schema.
- Remove processUncoveredFilesFromWhitelist attribute, useless (false
is its default value, and now have another name).
- Move from 4-spaces indented XML to 2-spaces indented.
- Small linefeed tweaks to generate better-looking XML.
Custom autoloaders are deprecated with PHPUnit 9 and will be removed
with PHPUnit 10.
Since PHPUnit 8.5 custom autoloaders don't do much because that
version removed the ability to launch unit tests by class name
and that's exactly the reason we had a custom autoloader (to map
class names to files within our tests). See MDL-67673 about
when direct use of classes was deprecated (8.5), now removed (9.5).
So, as far as it's unused, removing it now, test still can be
selectively using any of:
- a relative path to file (although there are some restrictions comming
with PHPUnit 9, see https://github.com/sebastianbergmann/phpunit/issues/4105
- using --filter, to point to any classname[::method]
- using --testsuite to run a complete suite
- using --config to point to custom components.
Also, commented out the lib/ajax/tests directory because it doesn't
exist / is empty and PHPUnit 9 emits error when a configured test
directory does not exist. See
https://github.com/sebastianbergmann/phpunit/issues/4493.
Alternative was to completely remove the configuration line, but
decided to keep it around in case some day we want to add some
test there.
Used by our custom assertTag() and assertNotTag() assertions, that some
day we should deprecate... the loading of XML content for further
processing has been moved to new classes within the PHPUnit utils. We
are just following the move here.
There are some unexpected differences between 310 and master,
better reconcile them now. Note this is not 100% critical but,
as far as nothing exclusively for 4.0 has landed to master yet
only branch/version differences should exist.
That custom printer was using some hacks to be able to intercept
configuration switches, reflection and other tricks to be able to
print the:
To re-run:
vendor/bin/phpunit --verbose "core_setuplib_testcase" lib/tests/setuplib_test.php
line on every failed/exceptional/skipped test. After some internal
discussion it was agreed that the normal phpunit output:
1) core_setuplib_testcase::test_localcachedir
Time is lower that allowed start value
Failed asserting that 1601976686 is equal to 1601976687 or is greater than 1601976687.
/var/www/html/lib/phpunit/classes/advanced_testcase.php:446
/var/www/html/lib/tests/setuplib_test.php:170
/var/www/html/lib/phpunit/classes/advanced_testcase.php:80
has already all the information at hand about how to rerun a test:
- vendor/bin/phpunit lib/tests/setuplib_test.php
- vendor/bin/phpunit --filter core_setuplib_testcase::test_localcachedir
- vendor/bin/phpunit --filter ::test_localcachedir
- vendor/bin/phpunit --testsuite componentname_testsuite
- vendor/bin/phpunit --config <<compoenent directory>>
- use --cache-result to get failed tests rerun with ease.
- ...
So better, let's use standard phpunit output and done. Also, note that,
with the upgrade to phpunit 8.5, the printer was not working correctly
any more, causing some switches, like --verbose ... to be ignored. Sure
it could have been fixed but, as commented above, no real need for all
that "parapheranlia" to print the rerun information.
Apply the standard deprecation procedure to the old APIs, to
be removed in Moodle 4.2 (MDL-69882). Existing unit tests
cover the deprecation debugging.
Originally MDL-64600
In charge of taking over the, now removed/archived, phpunit/dbunit
package. It supports loading of CSV/XML files and strings and
PHP arrays, allowing to send the loaded information to database.
Perform some basic controls about the consistency of information,
surely not super-exhaustive but fair enough.
100% covered with unit test.
Planned like an "interim" replacement for phpunit/dbunit uses in
core that, ideally, should be moved to generators stuff at some
point.
Note, expect a few tests in core to fail with this commit, I've
changed some fixtures around. Next commit will fix existing uses.
Originally MDL-64600
Constraints are now declared final, so we cannot extend them anymore.
Hence, instead of extending PHPUnit\Framework\Constraint\IsEqual we
are just wrapping it into our constraint.
All the setup/teardown/pre/post/conditions template methods
now are required to return void. This was warned with phpunit 7
and now is enforced.
At the same time, fix a few wrong function names,
provider data and param types, return statements...
We are already cleaning all sort of caches, statics, singletons
there and it's better to ensure they are always reset to avoid
myterious failures @ distance.
Andrew Nicols