improve logging: add dates to verbose stderr, match file log level to verbosity (#1772)
## Summary - **Add ISO date to verbose stderr format**: when running with `-v`, the stderr timestamp changes from `HH:mm:ss.SSS` to `YYYY-MM-DD HH:mm:ss.SSS`, matching the file log format. This makes it possible to correlate entries across days when stderr is captured by launchd, systemd, Docker, or file redirection. - **File log respects verbosity**: `exo.log` now uses DEBUG level when `-v` is passed, so the persistent log has the same detail as stderr. Previously it was always INFO regardless of verbosity. - **Startup banner with PID**: adds a visual separator and process ID to the startup message, making it easy to identify session boundaries in long-running logs (e.g. `grep "Starting EXO"`). The non-verbose (default) stderr format is unchanged — end-user terminal experience is not affected. ## Motivation When exo runs as a service (launchd, systemd, Docker), stderr is typically captured to a file. Without calendar dates in the timestamp, it is impossible to tell which day a log entry belongs to. This caused misidentification of log entries during a multi-day RDMA debugging session on a 4-node cluster. The file log (`exo.log`) already had dates and rotation, but only captured INFO level — missing the DEBUG output needed for postmortem analysis. ## Test plan - [x] `basedpyright` — 0 errors, 0 warnings, 0 notes - [x] `ruff check` — all checks passed - [x] `pytest` — 249 passed, 1 skipped Co-authored-by: user <[email protected]>
This commit is contained in:
+3
-1
@@ -264,7 +264,9 @@ def main():
|
||||
mp.set_start_method("spawn", force=True)
|
||||
# TODO: Refactor the current verbosity system
|
||||
logger_setup(EXO_LOG, args.verbosity)
|
||||
logger.info("Starting EXO")
|
||||
logger.info(f"{'=' * 40}")
|
||||
logger.info(f"Starting EXO | pid={os.getpid()}")
|
||||
logger.info(f"{'=' * 40}")
|
||||
logger.info(f"EXO_LIBP2P_NAMESPACE: {os.getenv('EXO_LIBP2P_NAMESPACE')}")
|
||||
|
||||
if args.offline:
|
||||
|
||||
@@ -66,7 +66,7 @@ def logger_setup(log_file: Path | None, verbosity: int = 0):
|
||||
else:
|
||||
logger.add(
|
||||
sys.__stderr__, # type: ignore
|
||||
format="[ {time:HH:mm:ss.SSS} | <level>{level: <8}</level> | {name}:{function}:{line} ] <level>{message}</level>",
|
||||
format="[ {time:YYYY-MM-DD HH:mm:ss.SSS} | <level>{level: <8}</level> | {name}:{function}:{line} ] <level>{message}</level>",
|
||||
level="DEBUG",
|
||||
colorize=True,
|
||||
enqueue=True,
|
||||
@@ -76,7 +76,7 @@ def logger_setup(log_file: Path | None, verbosity: int = 0):
|
||||
logger.add(
|
||||
log_file,
|
||||
format="[ {time:YYYY-MM-DD HH:mm:ss.SSS} | {level: <8} | {name}:{function}:{line} ] {message}",
|
||||
level="INFO",
|
||||
level="DEBUG" if verbosity > 0 else "INFO",
|
||||
colorize=False,
|
||||
enqueue=True,
|
||||
rotation=lambda _, __: next(rotate_once),
|
||||
|
||||
Reference in New Issue
Block a user