指南針解決方案

運用生成式 AI 建構採用虛擬服務專員技術的旅遊規劃應用程式

下圖顯示在整體架構中,您可以瞭解如何結合一流的 Google 開發人員服務,打造 AI 驅動應用程式。

建構精美、採用 AI 技術且適用於行動與網頁的應用程式

您可以使用 Flutter 和 Firebase Genkit 建構與 AI 完美整合的多平台應用程式。
您可以透過 Genkit,定義可驗證 LLM 輸出內容的結構定義,讓應用程式安心使用大型語言模型的資料。在 Flutter 中,您可以使用 Dart 將要求序列化,並透過標準 HTTP 要求將回應反序列化,以符合 Genkit 結構定義。
透過 IDX,您無須安裝任何軟體即可建立 Flutter 應用程式。如此一來,就能在瀏覽器中開發及測試 Android 應用程式和網頁應用程式。
Firebase Data Connect 是適用於行動和網頁應用程式的關聯資料庫服務,可讓你透過採用 Cloud SQL 技術的全代管 PostgreSQL 資料庫,執行建構作業及調度資源。此 API 使用能與 Firebase 驗證功能完美整合的 GraphQL,提供安全的結構定義、查詢和異動管理。Data Connect 包含針對 Kotlin Android 和網頁版的 SDK 支援。

使用 Firebase Genkit 建構代理程式

虛擬服務專員會使用 AI 自動化調度管理架構,接收使用者輸入內容及產生回覆。
---
model: googleai/gemini-1.5-flash-latest
config:
  temperature: 1.0
  safetySettings:
    - category: HARM_CATEGORY_HATE_SPEECH
      threshold: BLOCK_LOW_AND_ABOVE
    - category: HARM_CATEGORY_DANGEROUS_CONTENT
      threshold: BLOCK_ONLY_HIGH
    - category: HARM_CATEGORY_HARASSMENT
      threshold: BLOCK_LOW_AND_ABOVE
    - category: HARM_CATEGORY_SEXUALLY_EXPLICIT
      threshold: BLOCK_LOW_AND_ABOVE
input:
  schema:
    request: string, The users request for where they want to travel to.
    place: string, The place that closely represents the users request.
    placeDescription: string, A description of that place.
    activities(array, a stringify list of activities that can be found at the specified place): string
    restaurants?(array, a stringify list of all the restaurants found at that location): string
output:
  schema:
    place: string, The place the user is traveling to.
    itineraryName: string, a catchy itinerary name that encapsulates the spirit of the trip and includes the place name
    startDate: string, the start date of the trip
    endDate: string, the end date of the trip
    tags(array, relevant tags for the trip): string
    itinerary(array):
      day: number
      date: string
      planForDay(array):
        activityRef: string, the reference value for the activity - this comes from the available activities JSON. If no value is present use a ref value of restaurant.
        activityTitle: string, a catchy title for the activity
        activityDesc: string, a six word description of the activity
        photoUri?: string, set the photo uri value for restaurants only.
        googleMapsUri?: string, if this is a restaurant include the googleMapsUri
---

Generate an itinerary for a tourist planning on traveling to the location specified based in their request.
If there is something that does not exist within the list of activities, do not include it in your answer.
Feel free to relate the activitiy to the request in a meaningful way.
In the plan for day array, put activities as a travel brouchure might do.
Come up with a catchy name for the itinerary.

Pick three activities per day, minimum of three day trip unless otherwise specified in the request.

Output schema should not include the properties type or object.

Pick a date after 2024-05-14 but before 2024-12-31.

The output date must be in the format year-month-day.

Give each activity a unique title and description.

Limit activity descriptions to 6 words.

If no restaurants are supplied, do not recommend any restaurants to eat at.

{{#if restaurants}}
Find a restaurant to eat at each day.

Include a restaurant to visit in the itinerary for each day from the available restaurants.
The restaurant should be the only activity with a photoUri.
The photoUri for the restaurant should be from the photoUri property from the restaurant array.
If there are no restaurants to pick from, do not include it in the list.

The photoUri from the restaurantFinder should be in the format of places/${placeId}/photos/${photoId}

Each restaurant should be unique to the overall itinerary.
Each restaurant must contain a photoUri in their output JSON schema.
Each restaurant must also include  an activitiyRef, activityTitle, and activityDesc in their output
{{/if}}
Output must be in JSON format.

REQUEST : {{request}}
PLACE : {{place}}
PLACE DESCRIPTION : {{placeDescription}}
AVAILABLE ACTIVITIES : {{activities}}
RESTAURANTS : {{restaurants}}
使用生成式 AI 時,請務必設計有效的提示,模型才能傳回高品質的回覆。Firebase Genkit 提供 Dotprompt 外掛程式和文字格式,可協助你撰寫及管理生成式 AI 提示。此格式將提示、輸入和輸出結構定義、模型和設定全部封裝在單一檔案。

以下程式碼範例是旅遊應用程式中使用的 Dotprompt 檔案。這個結構定義是根據使用者在描述夢想行程時提供的資訊所產生。
Dotprompt 的設計是以程式碼為設計宗旨的幕後花絮。您可以在名為 dotprompt 檔案這種特別格式的檔案中撰寫及維護提示,使用與程式碼相同的版本管控系統追蹤這些內容的變更,然後部署這些內容,再搭配呼叫生成式 AI 模型的程式碼。
資料流是強型別、串流、本機和遠端呼叫的函式,且可完整觀測。Firebase Genkit 提供指令列介面和開發人員 UI,可用於處理流程,例如執行流程或對流程進行偵錯。
import {defineTool} from '@genkit-ai/ai/tool';
...
{
  name: 'restaurantFinder',
  description: `Used when needing to find a restaurant based on a users location.
  The location should be used to find nearby restaurants to a place. You can also
  selectively find restaurants based on the users preferences, but you should default
  to 'Local' if there are no indications of restaurant types in the users request.
  `,
  inputSchema: z.object({
    place: z.string(),
    typeOfRestaurant: z.string().optional() }),
    outputSchema: z.unknown(),
},
...
async (input) => {
  if (input.typeOfRestaurant == undefined) {
    input.typeOfRestaurant = "Local";
  }
  const geocodeEndpoint = "https://places.googleapis.com/v1/places:searchText";
  const textQuery = {textQuery: `${input.typeOfRestaurant} restaurants in ${input.place}`};

  const  response = await axios.post(
    geocodeEndpoint,
    JSON.stringify(textQuery),
    {
      headers: {
        "Content-Type": "application/json",
        "X-Goog-Api-Key": MAPS_API_KEY,
        "X-Goog-FieldMask": "places.displayName,places.formattedAddress,places.priceLevel,places.photos.name,places.editorialSummary,places.googleMapsUri"
      }
    }
  );
  console.log(response.data);
  let data = (response.data as PlaceResponse);
  for(let i = 0; i < data.places.length; i++) {
    if (data.places[i].photos) {
      data.places[i].photos = [data.places[i].photos[0]];
    }
  }
  return data as PlaceResponse;
}
您可以在 Genkit 中使用函式呼叫來擴充代理程式的功能,讓代理程式能進一步修正回應並完成其他工作。該旅遊應用程式定義了一項工具,可根據使用者想的行程,從 Places API 傳回餐廳資訊。程式碼會使用 Zod 定義輸入和輸出結構定義,以便驗證查詢結果。
...
export const textRefinement = defineFlow(
{
  name: 'textRefinement',
  inputSchema: z.string(),
  outputSchema: z.unknown(),
},
async (userRequest) => {
  const refinementPrompt = await prompt('textRefinement')
  const result = await refinementPrompt.generate({
      input: {
          request: userRequest
      },
  });
  return result.output();
});
為協助使用者提供更符合需求的搜尋體驗,在使用者描述夢想行程之後,Gemini 會根據旅遊應用程式提供的提示,判斷是否需要更多資訊,並在他們認為需要更多資訊時向應用程式發出信號。接著,應用程式會提示使用者輸入該項資訊,並附加到後端的要求中。
import 'package:http:http.dart' as http;
...
Future<List<Trip>> generateTrips(String description, List<Image> images) async {
  final uri = Uri.parse('.../generateTrips');
  final request = http.MultipartRequest('POST', uri);
  request.fields['description'] = description;
  request.files.add(http.MultipartFile.fromData(
      images.name, images.bytes,
      contentType: MediaType('image', 'png'),
  ));
  var response = await request.send();
  if (response.statusCode == 200) {
      final body = await response.body.text();
      final items = jsonDecode(body) as List<dynamic>;
      return items.map(Trip.fromJson).toList();
  }
  ...
  import { imagen2, geminiProVision } from '@genkit-ai/vertexai';
  import { generate } from '@genkit-ai/ai';

  const imageResult = await generate({
    model: imagen2,
    prompt: 'Generate an image of a very specific historical time and place.',
  });
  const generatedImage = imageResult.media();

  const descriptionResult = await generate({
    model: geminiProVision,
    prompt: [
      {
        text: 'What is the historical time and place represented in this picture?',
      },
      { media: generatedImage },
    ],
  });
  console.log(descriptionResult.text());
  }
旅遊應用程式會透過文字輸入功能,或是輕觸麥克風按鈕啟動語音轉文字,讓使用者定義自己的夢想行程。使用者也可以選擇上傳圖片。

應用程式利用 pub.dev 的 Dart 套件整合每個平台的原生語音轉文字功能,並使用 Firebase Genkit 中的 Gemini API 處理圖片或影片等多模態輸入內容。Gemini API 會使用檢索增強生成 (RAG) 功能,透過 Firebase Data Connect 和嵌入功能傳回一組建議行程,執行最鄰近搜尋

擴充正式版應用程式

Firebase 託管功能整合了 Flutter 等熱門的現代網路架構。只要搭配這些架構使用 Firebase 託管和 Cloud Functions for Firebase,您就能在偏好的架構環境中開發應用程式和微服務,然後在安全的代管伺服器環境中部署這些內容。在發布正式版前,請先瞭解應用程式中所有服務的安全性和效能。詳情請參閱 Firebase 發布檢查清單。
旅遊應用程式使用 Google AI 技術快速疊代測試資料,非常適合在 AI 用途極少,不需要擴充規模的應用程式時使用。Vertex AI 提供更多配額,可協助調度實際工作環境中的應用程式,並制定更嚴格的隱私權政策來保護使用者資料。Genkit 內建可輕鬆切換模型的功能,因此你不必重新編寫提示或 API 呼叫。