Skip to content

Commit 9e99933

Browse files
eavanvalkenburgeavanvalkenburg
authored
Python: new pre-commit actions and pre-commit as GHA (microsoft#6376)
### Motivation and Context <!-- Thank you for your contribution to the semantic-kernel repo! Please help reviewers and future users, providing the following information: 1. Why is this change required? 2. What problem does it solve? 3. What scenario does it contribute to? 4. If it fixes an open issue, please link to the issue here. --> - Added static code checking to pre-commit (check-ast and nbqa-check-ast) - Added Bandit security checking to pre-commit - Added pre-commit step to python-lint workflow, if it works, can delete seperate mypy, ruff and black tests ### Description <!-- Describe your changes, the overall approach, the underlying design. These notes will help understanding how your code works. Thanks! --> ### Contribution Checklist <!-- Before submitting this PR, please make sure: --> - [x] The code builds clean without any errors or warnings - [x] The PR follows the [SK Contribution Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md) and the [pre-submission formatting script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts) raises no violations - [x] All unit tests pass, and I have added new tests where possible - [x] I didn't break anyone 😄
1 parent d8674bd commit 9e99933

190 files changed

Lines changed: 2088 additions & 2173 deletions

File tree

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

‎.github/workflows/python-lint.yml‎

Lines changed: 5 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -7,67 +7,22 @@ on:
77
- 'python/**'
88

99
jobs:
10-
ruff:
10+
pre-commit:
1111
if: '!cancelled()'
1212
strategy:
1313
fail-fast: false
1414
matrix:
1515
python-version: ["3.10"]
1616
runs-on: ubuntu-latest
17-
timeout-minutes: 5
17+
continue-on-error: true
1818
steps:
19-
- run: echo "/root/.local/bin" >> $GITHUB_PATH
2019
- uses: actions/checkout@v4
2120
- name: Install poetry
2221
run: pipx install poetry
2322
- uses: actions/setup-python@v5
2423
with:
2524
python-version: ${{ matrix.python-version }}
2625
cache: "poetry"
27-
- name: Install Semantic Kernel
28-
run: cd python && poetry install --no-ansi
29-
- name: Run ruff
30-
run: cd python && poetry run ruff check .
31-
black:
32-
if: '!cancelled()'
33-
strategy:
34-
fail-fast: false
35-
matrix:
36-
python-version: ["3.10"]
37-
runs-on: ubuntu-latest
38-
timeout-minutes: 5
39-
steps:
40-
- run: echo "/root/.local/bin" >> $GITHUB_PATH
41-
- uses: actions/checkout@v4
42-
- name: Install poetry
43-
run: pipx install poetry
44-
- uses: actions/setup-python@v5
45-
with:
46-
python-version: ${{ matrix.python-version }}
47-
cache: "poetry"
48-
- name: Install Semantic Kernel
49-
run: cd python && poetry install --no-ansi
50-
- name: Run black
51-
run: cd python && poetry run black --check .
52-
mypy:
53-
if: '!cancelled()'
54-
strategy:
55-
fail-fast: false
56-
matrix:
57-
python-version: ["3.10"]
58-
runs-on: ubuntu-latest
59-
timeout-minutes: 5
60-
steps:
61-
- run: echo "/root/.local/bin" >> $GITHUB_PATH
62-
- uses: actions/checkout@v4
63-
- name: Install poetry
64-
run: pipx install poetry
65-
- uses: actions/setup-python@v5
66-
with:
67-
python-version: ${{ matrix.python-version }}
68-
cache: "poetry"
69-
- name: Install Semantic Kernel
70-
run: cd python && poetry install --no-ansi
71-
- name: Run mypy
72-
run: cd python && poetry run mypy -p semantic_kernel --config-file=mypy.ini
73-
26+
- name: Install dependencies
27+
run: cd python && poetry install
28+
- uses: pre-commit/action@v3.0.1

‎.github/workflows/python-test-coverage.yml‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,6 @@ jobs:
1010
python-tests-coverage:
1111
name: Create Test Coverage Messages
1212
runs-on: ${{ matrix.os }}
13-
continue-on-error: true
1413
permissions:
1514
pull-requests: write
1615
contents: read
@@ -21,14 +20,17 @@ jobs:
2120
os: [ubuntu-latest]
2221
steps:
2322
- name: Wait for unit tests to succeed
23+
continue-on-error: true
2424
uses: lewagon/wait-on-check-action@v1.3.4
2525
with:
2626
ref: ${{ github.event.pull_request.head.sha }}
2727
check-name: 'Python Unit Tests (${{ matrix.python-version}}, ${{ matrix.os }})'
2828
repo-token: ${{ secrets.GH_ACTIONS_PR_WRITE }}
2929
wait-interval: 10
30+
allowed-conclusions: success
3031
- uses: actions/checkout@v4
3132
- name: Download coverage
33+
continue-on-error: true
3234
uses: dawidd6/action-download-artifact@v3
3335
with:
3436
name: python-coverage-${{ matrix.os }}-${{ matrix.python-version }}.txt
@@ -37,6 +39,7 @@ jobs:
3739
search_artifacts: true
3840
if_no_artifact_found: warn
3941
- name: Download pytest
42+
continue-on-error: true
4043
uses: dawidd6/action-download-artifact@v3
4144
with:
4245
name: pytest-${{ matrix.os }}-${{ matrix.python-version }}.xml
@@ -45,6 +48,7 @@ jobs:
4548
search_artifacts: true
4649
if_no_artifact_found: warn
4750
- name: Pytest coverage comment
51+
continue-on-error: true
4852
id: coverageComment
4953
uses: MishaKav/pytest-coverage-comment@main
5054
with:

‎.pre-commit-config.yaml‎

Lines changed: 26 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,23 +7,37 @@ repos:
77
- id: sync_with_poetry
88
args: [--config=.pre-commit-config.yaml, --db=python/.conf/packages_list.json, python/poetry.lock]
99
- repo: https://github.com/pre-commit/pre-commit-hooks
10-
rev: v4.0.1
10+
rev: v4.6.0
1111
hooks:
1212
- id: check-toml
1313
files: \.toml$
1414
- id: check-yaml
1515
files: \.yaml$
16+
- id: check-json
17+
files: \.json$
18+
exclude: ^python\/\.vscode\/.*
1619
- id: end-of-file-fixer
1720
files: \.py$
1821
- id: mixed-line-ending
1922
files: \.py$
20-
- repo: https://github.com/psf/black
21-
rev: 24.4.2
23+
- id: debug-statements
24+
files: ^python\/semantic_kernel\/.*\.py$
25+
- id: check-ast
26+
name: Check Valid Python Samples
27+
types: ["python"]
28+
- repo: https://github.com/nbQA-dev/nbQA
29+
rev: 1.8.5
2230
hooks:
23-
- id: black
24-
files: \.py$
31+
- id: nbqa-check-ast
32+
name: Check Valid Python Notebooks
33+
types: ["jupyter"]
34+
- repo: https://github.com/asottile/pyupgrade
35+
rev: v3.15.2
36+
hooks:
37+
- id: pyupgrade
38+
args: [--py310-plus]
2539
- repo: https://github.com/astral-sh/ruff-pre-commit
26-
rev: v0.4.4
40+
rev: v0.4.5
2741
hooks:
2842
- id: ruff
2943
args: [ --fix, --exit-non-zero-on-fix ]
@@ -36,3 +50,9 @@ repos:
3650
language: system
3751
types: [python]
3852
pass_filenames: false
53+
- repo: https://github.com/PyCQA/bandit
54+
rev: 1.7.8
55+
hooks:
56+
- id: bandit
57+
args: ["-c", "python/pyproject.toml"]
58+
additional_dependencies: [ "bandit[toml]" ]

‎python/.vscode/extensions.json‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,9 @@
22
// See https://go.microsoft.com/fwlink/?LinkId=827846
33
// for the documentation about the extensions.json format
44
"recommendations": [
5-
"littlefoxteam.vscode-python-test-adapter",
65
"streetsidesoftware.code-spell-checker",
76
"ms-python.python",
7+
"charliermarsh.ruff",
8+
"rodolphebarbanneau.python-docstring-highlighter"
89
]
910
}

‎python/.vscode/settings.json‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,7 +18,6 @@
1818
],
1919
"python.testing.unittestEnabled": false,
2020
"python.testing.pytestEnabled": true,
21-
"pythonTestExplorer.testFramework": "pytest",
2221
"[python]": {
2322
"editor.codeActionsOnSave": {
2423
"source.organizeImports": "explicit",

‎python/.vscode/tasks.json‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,8 +93,7 @@
9393
"command": "poetry",
9494
"args": [
9595
"install",
96-
"--extras",
97-
"all"
96+
"--all-extras"
9897
],
9998
"presentation": {
10099
"reveal": "silent",

0 commit comments

Comments
 (0)