Skip to content

Commit 6e2597f

Browse files
ErotemicErotemic
committed
Merge branch 'dev/0.1.19' into 'main'
Start branch for dev/0.1.19 See merge request computer-vision/cmd_queue!21
2 parents 4f5de00 + dac9267 commit 6e2597f

48 files changed

Lines changed: 813 additions & 210 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.gitignore‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -94,3 +94,6 @@ ENV/
9494
profile_output.*
9595
_skbuild
9696
pip-wheel-metadata/
97+
98+
docs/source/_static/
99+
docs/source/images/

‎.gitlab-ci.yml‎

Lines changed: 41 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22

33
stages:
44
# TEMPLATES
5+
- lint
56
- build
67
- test
78
- gpgsign
@@ -30,9 +31,11 @@ stages:
3031
stage: build
3132
before_script:
3233
- 'python -V # Print out python version for debugging'
34+
- df -h
3335
script:
34-
- python -m pip install setuptools>=0.8 wheel build
36+
- python -m pip install setuptools>=0.8 wheel build twine
3537
- python -m build --wheel --outdir dist
38+
- python -m twine check ./dist/cmd_queue*.whl
3639
artifacts:
3740
paths:
3841
- ./dist/cmd_queue*.whl
@@ -370,9 +373,42 @@ test/full-strict/cp311-linux-x86_64:
370373
image: python:3.11
371374
needs:
372375
- build/cp311-linux-x86_64
376+
build/cp312-linux-x86_64:
377+
<<: *build_template
378+
image: python:3.12
379+
test/minimal-loose/cp312-linux-x86_64:
380+
<<: *test_minimal-loose_template
381+
image: python:3.12
382+
needs:
383+
- build/cp312-linux-x86_64
384+
test/full-loose/cp312-linux-x86_64:
385+
<<: *test_full-loose_template
386+
image: python:3.12
387+
needs:
388+
- build/cp312-linux-x86_64
389+
test/minimal-strict/cp312-linux-x86_64:
390+
<<: *test_minimal-strict_template
391+
image: python:3.12
392+
needs:
393+
- build/cp312-linux-x86_64
394+
test/full-strict/cp312-linux-x86_64:
395+
<<: *test_full-strict_template
396+
image: python:3.12
397+
needs:
398+
- build/cp312-linux-x86_64
399+
lint:
400+
<<: *common_template
401+
image: python:3.11
402+
stage: lint
403+
before_script:
404+
- df -h
405+
script:
406+
- pip install -r requirements/linting.txt
407+
- ./run_linter.sh
408+
allow_failure: true
373409
gpgsign/wheels:
374410
<<: *common_template
375-
image: python:3.8
411+
image: python:3.11
376412
stage: gpgsign
377413
artifacts:
378414
paths:
@@ -399,6 +435,8 @@ gpgsign/wheels:
399435
artifacts: true
400436
- job: build/cp311-linux-x86_64
401437
artifacts: true
438+
- job: build/cp312-linux-x86_64
439+
artifacts: true
402440
script:
403441
- ls dist
404442
- export GPG_EXECUTABLE=gpg
@@ -435,7 +473,7 @@ gpgsign/wheels:
435473
- ls dist
436474
deploy/wheels:
437475
<<: *common_template
438-
image: python:3.8
476+
image: python:3.11
439477
stage: deploy
440478
only:
441479
refs:

‎.mailmap‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Jon Crall <jon.crall@kitware.com> joncrall <erotemic@gmail.com>
2+
Jon Crall <jon.crall@kitware.com> joncrall <jon.crall@kitware.com>

‎.readthedocs.yml‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,11 +7,14 @@
77

88
# Required
99
version: 2
10+
build:
11+
os: "ubuntu-22.04"
12+
tools:
13+
python: "3.11"
1014
sphinx:
1115
configuration: docs/source/conf.py
1216
formats: all
1317
python:
14-
version: 3.7
1518
install:
1619
- requirements: requirements/docs.txt
1720
- method: pip

‎CHANGELOG.md‎

Lines changed: 7 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,13 @@ We are currently working on porting this changelog to the specifications in
55
This project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
66

77

8-
## Version 0.1.18 - Unreleased
8+
## Version 0.1.19 - Unreleased
9+
10+
### Fixed
11+
* Fixed issue with single-argv commands in the bash interface
12+
13+
14+
## Version 0.1.18 - Released 2023-08-09
915

1016
### Changed
1117

‎README.rst‎

Lines changed: 11 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,9 @@
11
Command Queue - cmd_queue
22
=========================
33

4-
.. .. |GitlabCIPipeline| |GitlabCICoverage| |Appveyor| |Codecov|
4+
.. .. |Appveyor| |Codecov|
55
6-
|Pypi| |Downloads| |ReadTheDocs|
6+
|Pypi| |Downloads| |GitlabCIPipeline| |GitlabCICoverage| |ReadTheDocs|
77

88

99
+------------------+-------------------------------------------------------------------------------------+
@@ -433,20 +433,20 @@ This prints the very simple slurm submission script:
433433
:target: https://cmd-queue.readthedocs.io/en/release/
434434

435435
.. # See: https://ci.appveyor.com/project/jon.crall/cmd_queue/settings/badges
436-
.. |Appveyor| image:: https://ci.appveyor.com/api/projects/status/py3s2d6tyfjc8lm3/branch/master?svg=true
437-
:target: https://ci.appveyor.com/project/jon.crall/cmd_queue/branch/master
436+
.. |Appveyor| image:: https://ci.appveyor.com/api/projects/status/py3s2d6tyfjc8lm3/branch/main?svg=true
437+
:target: https://ci.appveyor.com/project/jon.crall/cmd_queue/branch/main
438438

439-
.. |GitlabCIPipeline| image:: https://gitlab.kitware.com/utils/cmd_queue/badges/master/pipeline.svg
440-
:target: https://gitlab.kitware.com/utils/cmd_queue/-/jobs
439+
.. |GitlabCIPipeline| image:: https://gitlab.kitware.com/computer-vision/cmd_queue/badges/main/pipeline.svg
440+
:target: https://gitlab.kitware.com/computer-vision/cmd_queue/-/jobs
441441

442-
.. |GitlabCICoverage| image:: https://gitlab.kitware.com/utils/cmd_queue/badges/master/coverage.svg?job=coverage
443-
:target: https://gitlab.kitware.com/utils/cmd_queue/commits/master
442+
.. |GitlabCICoverage| image:: https://gitlab.kitware.com/computer-vision/cmd_queue/badges/main/coverage.svg?job=coverage
443+
:target: https://gitlab.kitware.com/computer-vision/cmd_queue/commits/main
444444

445445
.. |CircleCI| image:: https://circleci.com/gh/Erotemic/cmd_queue.svg?style=svg
446446
:target: https://circleci.com/gh/Erotemic/cmd_queue
447447

448-
.. |Travis| image:: https://img.shields.io/travis/Erotemic/cmd_queue/master.svg?label=Travis%20CI
448+
.. |Travis| image:: https://img.shields.io/travis/Erotemic/cmd_queue/main.svg?label=Travis%20CI
449449
:target: https://travis-ci.org/Erotemic/cmd_queue
450450

451-
.. |Codecov| image:: https://codecov.io/github/Erotemic/cmd_queue/badge.svg?branch=master&service=github
452-
:target: https://codecov.io/github/Erotemic/cmd_queue?branch=master
451+
.. |Codecov| image:: https://codecov.io/github/Erotemic/cmd_queue/badge.svg?branch=main&service=github
452+
:target: https://codecov.io/github/Erotemic/cmd_queue?branch=main

‎cmd_queue/__init__.py‎

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,7 @@
11
r"""
2+
Command Queue
3+
=============
4+
25
The cmd_queue module is a tool that lets users define a DAG of bash commands.
36
This DAG can be executed in a lightweight tmux backend, or a heavyweight slurm
47
backend, or in simple serial mode that runs in the foreground thread. Rich
@@ -303,7 +306,7 @@
303306
__mkinit__ = """
304307
mkinit -m cmd_queue
305308
"""
306-
__version__ = '0.1.18'
309+
__version__ = '0.1.19'
307310

308311

309312
__submodules__ = {

‎cmd_queue/base_queue.py‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -117,6 +117,7 @@ def write(self):
117117
def submit(self, command, **kwargs):
118118
"""
119119
Args:
120+
command (str): The command to execute
120121
name: specify the name of the job
121122
**kwargs: passed to :class:`cmd_queue.serial_queue.BashJob`
122123
"""
@@ -157,6 +158,7 @@ def submit(self, command, **kwargs):
157158
raise
158159
job = serial_queue.BashJob(command, depends=depends, **kwargs)
159160
else:
161+
assert isinstance(command, Job)
160162
# Assume job is already a bash job
161163
job = command
162164
self.jobs.append(job)

‎cmd_queue/cli_boilerplate.py‎

Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -116,6 +116,39 @@ class CMDQueueConfig(scfg.DataConfig):
116116
:func:`CMDQueueConfig.run_queue`. In this way you do not need to worry
117117
about this specific boilerplate when writing your application. See
118118
``cmd_queue.cli_boilerplate __doc__:0`` for example usage.
119+
120+
It is a good idea to overwrite the default value of queue_name when
121+
inheriting: e.g.
122+
123+
.. code:: python
124+
125+
queue_name = scfg.Value('your_default_name', help='overwrite the default queue name', group='cmd-queue')
126+
127+
Other defaults that can be overwritten are:
128+
129+
.. code:: python
130+
131+
run = scfg.Value(False, isflag=True, help='if False, only prints the commands, otherwise executes them', group='cmd-queue')
132+
133+
backend = scfg.Value('tmux', help=('The cmd_queue backend. Can be tmux, slurm, or serial'), group='cmd-queue')
134+
135+
print_commands = scfg.Value('auto', isflag=True, help='enable / disable rprint before exec', group='cmd-queue')
136+
137+
print_queue = scfg.Value('auto', isflag=True, help='print the cmd queue DAG', group='cmd-queue')
138+
139+
with_textual = scfg.Value('auto', isflag=True, help='setting for cmd-queue monitoring', group='cmd-queue')
140+
141+
other_session_handler = scfg.Value('ask', help='for tmux backend only. How to handle conflicting sessions. Can be ask, kill, or ignore, or auto', group='cmd-queue')
142+
143+
virtualenv_cmd = scfg.Value(None, type=str, help=ub.paragraph(
144+
'''
145+
Command to start the appropriate virtual environment if your bashrc
146+
does not start it by default.'''), group='cmd-queue')
147+
148+
tmux_workers = scfg.Value(8, help='number of tmux workers in the queue for the tmux backend', group='cmd-queue')
149+
150+
slurm_options = scfg.Value(None, help='if the backend is slurm, provide a YAML dictionary for things like partition / etc...', group='cmd-queue')
151+
119152
"""
120153
run = scfg.Value(False, isflag=True, help='if False, only prints the commands, otherwise executes them', group='cmd-queue')
121154

‎cmd_queue/cli_boilerplate.pyi‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -27,5 +27,6 @@ class CMDQueueConfig(scfg.DataConfig):
2727

2828
def run_queue(config,
2929
queue: cmd_queue.Queue,
30-
print_kwargs: None | Dict = None) -> None:
30+
print_kwargs: None | Dict = None,
31+
**kwargs) -> None:
3132
...

0 commit comments

Comments
 (0)