Activities
·····
2025-12
fix(providers): handle Vertex AI global region endpoint
Resolves #6569 `region: global` → `aiplatform.googleapis.com` (no prefix) other regions → `{region}-aiplatform.googleapis.com` References: - [Vertex AI locations documentation](https://cloud.g...
Vertex AI provider returns 404 for region: global
**Describe the bug** The Vertex AI provider generates an incorrect API host when `region: global` is specified, resulting in 404 errors. The provider attempts to construct the hostname using the pat...
AI で再注目された技術やツールたち
これは はてなエンジニア Advent Calendar 2025 2 日目の記事です。 年の瀬なので振り返りたくなる季節ですね。 今日はそんな「以前から存在していたけど AI によって新しい価値を見出された技術やツールを語ろう」のコーナーです。 いくぞ!! Server-Sent Events (SSE) まず出世頭として思いつくのは Server-Sent Events でしょう。 Serve...
2025-11
Claiming namespace pokutuna
I have already published extensions to the VS Code Marketplace under the publisher name "pokutuna". I would like to publish to the Open VSX Registry as well, using the same publisher namespace "pokut...
松尾研LLM講座申し込み締切もうすぐ!! & 2024 年講座の思い出
大規模言語モデル講座 応用編 2025 Autumn - 東京大学松尾・岩澤研究室（松尾研）- Matsuo Lab 2025/11/19(水) AM10:00 まで!!! 社会人枠もあります。自分は前年に参加してとても良かったのでおすすめしています。 各社の LLM API を雰囲気で利用していて、理解が足りてないな、もどかしいなと感じて応募したのですが、そういった Web エンジニアの方々は...
fix(docker): resolve @swc/core SIGSEGV on Alpine Linux
resolves #6126 ## Summary This PR resolves the SIGSEGV error that has been blocking Docker releases. The issue was caused by a regression in `@swc/core@1.13.20` that causes segmentation faults ...
Docker images not updated since v0.118.14
**Describe the bug** The Docker release workflow has been failing consistently. As a result, the Docker image at https://ghcr.io/promptfoo/promptfoo has not been updated since version 0.118.14 (relea...
Feature Request: enabled option to switch enable/disable MCP servers
Thank you for creating this great tool. I'm using mcp-proxy to consolidate multiple MCP sedskrver configurations for multiple local AI tools. I'd like to propose adding an `enabled` boolean field to ...
2025-10
Feature Request: Export notebook with outputs as HTML
The VSCode extension currently lacks a feature to export notebooks with their execution outputs as HTML, equivalent to the "Download as HTML" button available in Web UI. The current workaround is to ...
JSON Schema validation errors for custom fields (assertionTemplates & $ref)
**Describe the bug** When using promptfooconfig.yaml with the YAML Language Server schema reference (`$schema=https://promptfoo.dev/config-schema.json`), VS Code and other editors display validation e...
fix(provider): merge config and prompt systemInstruction instead of throwing error in gemini
resolves #5822 Previously, when both config.systemInstruction and prompt system role existed, the Gemini provider would throw an error. This changes the behavior to merge both instructions with co...
Gemini: Support merging config and assertion systemInstructions using parts
**Describe the bug** The Gemini provider throws a "Template error: system instruction defined in prompt and config" error when both `config.systemInstruction` and assertion-added system messages are ...
2025-09
with_structured_output returns None when MALFORMED_FUNCTION_CALL happens without raising errors
When using `with_structured_output()`, occasionally the return value becomes `None` unexpectedly. This happens even with function calling that normally succeeds, where the Gemini API sometimes returns...
fix(webui): prioritize JSON prettify over Markdown rendering when both enabled
resolves #5704 This is necessary for use cases where we need to evaluate outputs that include both Markdown text for users and multiple structured data from an Agent. Markdown rendering (top)...
JSON Prettify option does not work when Markdown rendering is enabled in WebUI
**Describe the bug** JSON Prettify option does not work when Markdown rendering is enabled simultaneously in the WebUI settings. When both "Render Markdown content" and "Prettify JSON outputs" options...
fix(providers): handle system-only prompt in Gemini
resolves: #5501 I confirmed that the following config, which previously did not work, now works with this fix. https://gist.github.com/pokutuna/071b4812933e79022a058d1af5de6320 - `$ GEMINI_API_...
Some assertions in Gemini evaluation fail with system-only prompts
**Describe the bug** When using Google Gemini Test providers (both Google AI Studio and Vertex AI), specific evaluation assertions fail with a 400 Bad Request error. The affected assertions are thos...
Vertex AI (Google Cloud) provider ignores explicit projectId configuration
**Describe the bug** The Vertex AI provider ignores explicitly configured `projectId` values and `VERTEX_PROJECT_ID` environment variables, always using the project ID from Application Default Creden...
fix: prevent authentication warnings when tracing is disabled
resolves: https://github.com/langfuse/langfuse/issues/8978 Added early return in `Langfuse.__init__()` when tracing_enabled=False to skip authentication checks entirely. Only committed the specific ...
bug: Authentication warnings shown even when LANGFUSE_TRACING_ENABLED=false
### Describe the bug When setting `LANGFUSE_TRACING_ENABLED=false` to disable Langfuse tracing in the Python SDK, authentication warning messages are still displayed even though tracing is explicitly...