maxframe.learn.contrib.llm.multi_modal.generate#

maxframe.learn.contrib.llm.multi_modal.generate(data, model: MultiModalGenLLM, messages=None, prompt_template=None, simple_output: bool = False, params: Dict[str, Any] | None = None, **kw)[source]#

Generate text with a multimodal LLM from MaxFrame data.

Parameters:

  • data (DataFrame or Series) – Input data used to render one request per row. Template placeholders reference columns by name, for example "{image_url}".

  • model (MultiModalGenLLM) – Multimodal generation model instance.

  • messages (list of dict, optional) – Chat messages template. This is an alias of prompt_template and takes precedence when both arguments are provided.

  • prompt_template (str or list of dict, optional) – Prompt template used to build each request. A list value should follow chat message shape, where message content may contain plain text, provider-compatible dicts, or ContentPart values.

  • simple_output (bool, default False) – Whether to return the generated text directly when supported by the model executor, instead of the raw provider response.

  • params (dict, optional) – Additional generation parameters such as temperature or max tokens.

Returns:

A DataFrame with response and success columns. Failed requests store the error message in response.

Return type:

DataFrame

Examples

Build chat messages with text and image URL placeholders:

>>> from maxframe.learn.contrib.llm import ContentPart, ImageContentType
>>> from maxframe.learn.contrib.llm.models.dashscope import DashScopeMultiModalLLM
>>> import maxframe.dataframe as md
>>>
>>> df = md.DataFrame({"image_url": ["https://example.com/cat.png"]})
>>> model = DashScopeMultiModalLLM(
...     name="qwen-vl-max",
...     api_key_resource="<api-key-resource-name>",
... )
>>> messages = [
...     {
...         "role": "user",
...         "content": [
...             ContentPart.text("Analyze this image."),
...             ContentPart.image(
...                 data=df.image_url,
...                 type=ImageContentType.IMAGE_URL,
...             ),
...         ],
...     }
... ]
>>> result = model.generate(df, messages=messages)

Use an OSS object URL with explicit storage options when the image is not publicly reachable:

>>> storage_options = {
...     "access_key_id": "<access-key-id>",
...     "access_key_secret": "<access-key-secret>",
... }
>>> df = md.DataFrame({"image_url": ["oss://endpoint/bucket/path/cat.png"]})
>>> messages = [
...     {
...         "role": "user",
...         "content": [
...             ContentPart.text("Analyze this OSS image."),
...             ContentPart.image(
...                 data=df.image_url,
...                 type=ImageContentType.IMAGE_URL,
...                 storage_options=storage_options,
...             ),
...         ],
...     }
... ]
>>> result = model.generate(df, messages=messages)

Notes

ContentPart.image supports IMAGE_URL, BINARY and BASE64 image content. BINARY and BASE64 inputs must include mime_type.