Commit Graph

50 Commits

Author SHA1 Message Date
David Monllao 26b86f31e5 MDL-61028 search: Clear search engine users cache after tests 2018-04-24 15:48:40 +02:00
David Monllao c5a8065258 Merge branch 'MDL-61028-master' of https://github.com/sammarshallou/moodle 2018-04-23 13:09:53 +02:00
sam marshall 222a97ce76 MDL-61028 core_search: Allow filtering search by user (back-end, Solr)
Adds support in the Solr engine for a new 'userids' parameter to
restrict search results to those created by a specific user or users.
2018-04-20 13:36:55 +01:00
sam marshall 73fd56663b MDL-61102 core_search: Improve performance by reducing schema checks 2018-04-18 16:24:51 +01:00
sam marshall a539e5605c MDL-61824 core_search: Deleting restored context caused error
When restoring content, this adds it to a queue for indexing. If the
restored content was then deleted before the indexing takes place,
this caused an exception in the scheduled task.

This change makes it continue safely past missing contexts.
2018-04-09 16:36:30 +01:00
sam marshall fc440796e9 MDL-59434 core_search: Alternate result orders including by context
Implements a mechanism by which search engines can provide different
result orderings, and implements a 'by location' ordering within the
Solr search engine (available whenever the user starts their search
from within a course or activity).
2018-03-20 12:15:25 +00:00
sam marshall 4359ef18bb MDL-58885 core_search: Add group support
Adds group support to the core search API and the Solr search engine.
This allows for:

* User searching by group (in the API only, no interface yet)
* Automatically restrict search results by group (in some cases like
  separate-groups forums)
2018-02-16 16:57:00 +00:00
sam marshall 65da6840f8 MDL-60981 core_search: UI to gradually reindex a single area
Adds a new 'Gradual reindex' link to the search areas page for each
area. When clicked, this takes you to a confirm prompt, and then
adds each context from that search area to the indexing queue.

The search areas page now displays the 'Additional indexing queue'
(if it is non-empty). The table shows the first 10 items in the
queue, and it also indicates the total number in case there are
more. (I don't think people really need to see the entire
contents of it, so I didn't implement paging.)
2017-12-22 13:05:11 +00:00
sam marshall 8736fbc190 MDL-60981 core_search: New indexpriority field in search index queue
Adds indexpriority field to the database table which holds a queue of
indexing requests. This allows for potentially large area reindexes
to have a lower priority, so as not to halt the special indexes that
run after a course restore.
2017-12-22 13:05:10 +00:00
Damyon Wiese 97812f14c1 Merge branch 'MDL-60880-master' of https://github.com/sammarshallou/moodle 2017-12-06 19:53:37 +13:00
sam marshall cfa00fc565 MDL-60880 core_search: Allow search of specific context (back-end)
Adds back-end support for restricting searches to specified context
ids (for example so it is possible to search only a specific forum).
2017-12-05 17:42:38 +00:00
sam marshall 1b8cf12a59 MDL-60857 core_search: Improve cron log progress information 2017-12-01 15:47:22 +00:00
Adam Olley 7f83e99e5f MDL-60707 core_search: In tests, use faked time for indexing length
Without this change its possible that the unit tests will fail at any time.
Before this change the indexing time is measured by real-time, not fake time,
making all index timings 0.

This happens as PHP offers no guarantee around the sort-order of an array for
any given two members that equate as equal. It just happens to pass for the
current array of search areas in vanilla Moodle.
2017-11-09 10:49:53 +10:30
sam marshall 82735dec4d MDL-60555 core_search: Change unit tests to use fake time
Using real time caused rare failures. I've changed it to use fake
measurement for the indexing timeouts, which were unit-tested.
2017-10-31 14:48:09 +00:00
Andrew Nicols e61ad18e70 Merge branch 'MDL-55356-master' of https://github.com/sammarshallou/moodle 2017-10-12 11:28:04 +08:00
sam marshall 74b7a42d0d MDL-55356 core_search: Make indexing task/CLI do context requests 2017-10-11 17:17:10 +01:00
sam marshall 4ba11aa907 MDL-55356 core_search: New manager API index_context
New function which uses the get_document_recordset function in
search areas to support full and partial indexing of a given
context.
2017-10-11 17:17:10 +01:00
sam marshall eacb2bd11b MDL-55356 core_search: API to queue contexts for indexing
New API \core_search\manager::request_index($context, $areaid = '')
adds the given context to a list which is intended to be indexed
later by the scheduled task.
2017-10-11 17:17:07 +01:00
sam marshall d761b3fb19 MDL-55356 core_search: API to check if indexing enabled
New function \core_search\manager::is_indexing_enabled(), analagous
to existing is_global_search_enabled().

This replaces existing duplicated code, ready for more use in
following commits.
2017-10-11 17:17:06 +01:00
sam marshall 2d2fcc1c9e MDL-60357 core_search: Future modified times cause serious problems 2017-10-06 13:38:32 +01:00
sam marshall 2d94d4ea7c MDL-59913 Global search: Allow search of non-enrolled courses 2017-09-29 16:30:37 +01:00
sam marshall e36eefaeea MDL-59926 core_search: Allow Behat testing of results screens 2017-09-04 17:42:53 +01:00
David Monllao afabbd110e Merge branch 'MDL-59039-master' of https://github.com/sammarshallou/moodle 2017-07-11 21:49:03 +02:00
sam marshall 67d6479581 MDL-59039 Global search: Allow partial indexing (in scheduled task) 2017-07-11 16:41:33 +01:00
sam marshall a96faa49a8 MDL-58957 Global search: Add block support to search manager 2017-06-30 13:10:12 +01:00
Matt Porritt 0a9a10f029 MDL-58654 Global search: Allow engines to support batch processing.
Allow Global Search 3rd party engines to be much faster by supporting batch processing.
Refactor the iterator loop for the documents to add to the search index into its own method.
Move this new method from the manger class to the base engine class.
The move to the base engine class will allow search engine plugins to override this and determine
how they implment it. It will not break the existing interface contract with existing plugins.

Finally, add an elasped time indication to the trace output.
2017-05-31 08:28:12 +10:00
David Monllao b805d3f8ef MDL-56057 search: Fix incorrectly reported no enabled search areas
Also improved performance caching enabled search areas and
all search areas at the same time.
2016-10-12 08:30:23 +08:00
David Monllao e71061a22b MDL-55191 search: Generate an search result viewed event 2016-09-15 09:36:49 +08:00
David Monllao f3d38863fe MDL-55390 search: is_search_area unit tests
Added extra checking for search manager get_search_area.
2016-08-09 15:51:30 +08:00
Andrew Nicols 46c31069f3 MDL-55390 core_search: Correct variable name 2016-08-09 13:44:10 +08:00
Devang Gaur 396d6f0a6e MDL-55390 core_search: check if a search area is real 2016-08-05 01:12:21 +05:30
Dan Poltawski 773a11519d Merge branch 'MDL-55278_master' of https://github.com/dmonllao/moodle 2016-07-26 11:57:09 +01:00
David Monllao 379ca98642 MDL-55287 search: Check that an engine has been selected 2016-07-21 18:40:29 +02:00
David Monllao 0bd8383a8b MDL-55278 search: Updating references to area namespace 2016-07-21 18:38:18 +02:00
David Monllao 6a4c2146f9 MDL-53222 search: Replace report_search by new searchareas admin page 2016-07-11 11:49:14 +02:00
David Monllao 25ba053ff2 MDL-54707 search: Add user context to get_areas_user_accesses 2016-06-26 21:23:16 +02:00
Eloy Lafuente (stronk7) 33952d4a53 Merge branch 'MDL-53758-master' of git://github.com/merrill-oakland/moodle 2016-04-26 20:34:20 +02:00
David Monllao 23fc1be833 MDL-53535 search_solr: Solr version restrictions
Also removing is_server_ready call from execute_query as in production
workflows search is done through \core_search\manager::search and
\core_search\manager::instance already checks is_server_ready. In
testing environment setUp functions should check that the server is
ready before performing other actions.
2016-04-18 09:59:58 +08:00
Eric Merrill 053118a184 MDL-53758 search: Better results with low hit rates, improve performance
Ensures that Solr will return available results, even if there are many
misses when using check_access(), by asking Solr for more results if the
counter says there should be more.

Improves performance by ending processing as soon as the requested page
of results is processed. Remaining number of pages is an "estimate"
based on the total result count from Solr and how many items we have
rejected up to this point.
2016-04-17 21:59:25 -04:00
Eric Merrill 427e3cbcd6 MDL-53167 search: Add ability to limit courses searched 2016-04-07 14:41:59 -04:00
David Monllao 9470e4d1ed MDL-53272 search: Remove search results cache 2016-04-04 15:38:03 +08:00
Eric Merrill 091973dbd7 MDL-53515 search: Extend search API to allow file indexing 2016-03-24 15:44:27 -04:00
Eloy Lafuente (stronk7) 3edc162611 Merge branch 'MDL-53354-master' of git://github.com/merrill-oakland/moodle 2016-03-16 01:09:45 +01:00
Eric Merrill f6b425e2de MDL-53354 search: Add ability to limit search doc to a user 2016-03-10 09:19:05 -05:00
Andrew Nicols bd2e5ef17d Merge branch 'MDL-53325-master' of git://github.com/merrill-oakland/moodle 2016-03-09 10:06:47 +08:00
Eric Merrill 075fa91271 MDL-53325 search: Remove commit from engine interface
Remove commit from engine, and instead notify when indexes
start and stop, allowing them to decide what to do.
2016-03-06 13:46:54 -05:00
Eric Merrill bf2235bb11 MDL-53325 search: Optimize Solr less often 2016-03-06 13:46:54 -05:00
Dan Poltawski 7c7757185a MDL-53247 search: allow search to be configured before enabled
The seach manager no longer checks if search is enabled before providing
an instance. It's up to the callers to do the access control.
2016-03-03 12:31:21 +08:00
David Monllao 69d66020ce MDL-31989 search: get_config method to search areas
Also content_to_text moved to weblib.
2016-02-23 10:47:59 +00:00
David Monllao db48207e1a MDL-31989 search: Search API and search engine API
Introducing both APIs in moodle along with:
- search_box widget to add a tiny search box
- admin settings with setup steps helper
- cache for search results
- template for a search result
- php unit stuff

Many thanks to Tomasz Muras, Prateek Sachan and Daniel Neis for their contributions, for starting this development
and for pushing for it to be completed. Also thanks to other contributors: Jonathan Harker and eugeneventer.
2016-02-23 10:47:58 +00:00