Leveraging Orchestration Capabilities to Enhance Responses

[frankmjlao]

Tutorials: https://developers.sap.com/tutorials/ai-core-orchestration-consumption-opt.html

We found some block leading the python sdk script not able to enable the filter condition, some of them i can fix, but some of them not able to fix

1. How these AzureContentSafty class coming into the picture is not mentioned (not able to fixed)

input_filter= AzureContentSafety(hate=AzureThreshold.ALLOW_SAFE,
                                  violence=AzureThreshold.ALLOW_SAFE,
                                  self_harm=AzureThreshold.ALLOW_SAFE,
                                  sexual=AzureThreshold.ALLOW_SAFE)
input_filter_llama = LlamaGuard38bFilter(hate=True)
output_filter = AzureContentFilter(hate=AzureThreshold.ALLOW_SAFE,
                                   violence=AzureThreshold.ALLOW_SAFE_LOW,
                                   self_harm=AzureThreshold.ALLOW_SAFE_LOW_MEDIUM,
                                   sexual=AzureThreshold.ALLOW_ALL)
output_filter_llama = LlamaGuard38bFilter(hate=True)

2. when creating the orchestration service instance, missing the proxy_client parameter, i finally solve the issue with following code

from gen_ai_hub.orchestration.service import OrchestrationService 

from gen_ai_hub.proxy.core.proxy_clients import get_proxy_client
proxy_client = get_proxy_client('gen-ai-hub', ai_core_client=ai_core_client) # for an AI Core proxy

print("Proxy client obtained successfully.")

# Initialize an empty list to store the responses 
responses = [] 

# Iterate through each config and get the response using the filtered input 
for i, config in enumerate(configs): 
    orchestration_service = OrchestrationService(api_url=ready_deployment.deployment_url, 
                                                 config=config, 
                                                 proxy_client=proxy_client # this is not documented in the tutorial
                                                 ) 

3. SystemMessage and UserMessage class seems no longer available in template module, but moved into message modules, like below

from gen_ai_hub.orchestration.models.message import SystemMessage, UserMessage

Write here how you think we can improve the tutorial ...

[rbrainey]

Hi @I321506 - please have a look at this issue and respond. Thanks!