The VSCode extension currently lacks a feature to export notebooks with their execution outputs as HTML, equivalent to the "Download as HTML" button available in Web UI. The current workaround is to ...
**Describe the bug** When using promptfooconfig.yaml with the YAML Language Server schema reference (`$schema=https://promptfoo.dev/config-schema.json`), VS Code and other editors display validation e...
resolves #5822 Previously, when both config.systemInstruction and prompt system role existed, the Gemini provider would throw an error. This changes the behavior to merge both instructions with co...
**Describe the bug** The Gemini provider throws a "Template error: system instruction defined in prompt and config" error when both `config.systemInstruction` and assertion-added system messages are ...
When using `with_structured_output()`, occasionally the return value becomes `None` unexpectedly. This happens even with function calling that normally succeeds, where the Gemini API sometimes returns...
resolves #5704 This is necessary for use cases where we need to evaluate outputs that include both Markdown text for users and multiple structured data from an Agent. Markdown rendering (top)...
**Describe the bug** JSON Prettify option does not work when Markdown rendering is enabled simultaneously in the WebUI settings. When both "Render Markdown content" and "Prettify JSON outputs" options...
resolves: #5501 I confirmed that the following config, which previously did not work, now works with this fix. https://gist.github.com/pokutuna/071b4812933e79022a058d1af5de6320 - `$ GEMINI_API_...
**Describe the bug** When using Google Gemini Test providers (both Google AI Studio and Vertex AI), specific evaluation assertions fail with a 400 Bad Request error. The affected assertions are thos...
**Describe the bug** The Vertex AI provider ignores explicitly configured `projectId` values and `VERTEX_PROJECT_ID` environment variables, always using the project ID from Application Default Creden...
resolves: https://github.com/langfuse/langfuse/issues/8978 Added early return in `Langfuse.__init__()` when tracing_enabled=False to skip authentication checks entirely. Only committed the specific ...
### Describe the bug When setting `LANGFUSE_TRACING_ENABLED=false` to disable Langfuse tracing in the Python SDK, authentication warning messages are still displayed even though tracing is explicitly...
## TLDR Fixes incorrect positioning of citation markers in WebSearchTool when responses contain multibyte characters by properly handling byte positions from the Gemini API instead of treating them...
### What happened? The WebSearchTool inserts citation markers (e.g., `[1]`, `[2]`) at incorrect positions when the search response contains multibyte characters such as Japanese text. The Grounding A...
を作りました @pokutuna/mcp-chrome-tabs デモ AppleScript を使っている都合上 macOS でのみ動きます。 なぜ作ったか 既に LLM にブラウザを操作させる技術は色々ある。browser-use、playwright-mcp、mcp-chrome などなど。 これらの Tool, MCP は便利だけど、様々なツールが入っていてコンテキストへの圧迫が大きく、C...
