Starting a dev log for my hobby projects. Sharing ideas, and also any knowledge that is learned along the way.

Project Idea

For a recent side project, I decided to build a simple travel search application.

While I'm not the most frequent flyer, when I do travel, the search process ends up in a mini-optimization game between flight prices and hotel prices.

So, what ends up happening is that I spend time staring at a couple of graphs, like these:

One for flights and one for hotels, to see where the price point is optimal for the conditions I want to meet. Maybe sites like google travel or Expedia already have tools that do this for me, but I have not found the best solution for my particular needs. Even if there is a site out there that does this, I figured, it's probably a good learning exercise to make this into a side project!

Research

To build this app, I need a data source for querying price data for flights, hotels, etc. I've tried searching for free or reasonably priced APIs that I could use but did not find anything that looked promising. 😞

So my second option was to reverse-engineer and do a bit of web-scraping on existing travel sites. The price graphs and calendar view above are all from Google Travel (aka Flights), which contains all the information I would need, and also has the data publicly available (i.e. does not require a login or anything).

A bit of browser inspection shows me that every time I open the flight Price Graph UI, there is a call made to this GetCalendarGraph endpoint to fetch data. After a bit of testing, it looks like I can adjust the input query a bit to get the data I need.

Flight API

/**
 * Query price data for flight between a range of dates 
 * @param from - starting city code
 * @param to - destination city code
 * @param flight_dates - date of flight
 * @param date_range - date range for price data
 * @param isNotStopOnly - filter non-stop flights only?
 */
async function getFlightPrices(from: string, to: string, flight_dates: string[], date_range: string[], isNonStopOnly: boolean) {
    // need to calculate how long the trip is (in days)
    const dayTrip = new Date(flight_dates[1]).getDate() - new Date(flight_dates[0]).getDate();

    const flightData = [
        null,
        [null, null, 1, null, [], 1, [1, 0, 0, 0], null, null, null, null, null, null, [[[[[from, 0]]], [[[to, 0]]], null, (isNonStopOnly ? 1 : 0), [], [], flight_dates[0], null, [], [], [], null, null, [], 3], [[[[to, 4]]], [[[from, 4]]], null, (isNonStopOnly ? 1 : 0), [], [], flight_dates[1], null, [], [], [], null, null, [], 3]], null, null, null, true, null, null, null, null, null, [], null, null, null],
        date_range,
        null,
        [
            dayTrip,
            dayTrip
        ]
    ];

    const queryData = encodeURIComponent(JSON.stringify([null, JSON.stringify(flightData)]));
    const { data } = await axios({
        method: "POST",
        url: "https://www.google.com/_/TravelFrontendUi/data/travel.frontend.flights.FlightsFrontendService/GetCalendarGraph",
        data: `f.req=${queryData}`,
        headers: { "Content-Type": "application/x-www-form-urlencoded;charset=UTF-8" },

      return data;
});

// -------------------//
// Test code
// -------------------//
getFlightPrices(
    "SFO", "JFK",
    ["2023-11-01", "2023-11-05"], 
    ["2023-10-01", "2023-12-01"],
    true
).then(x => {
    console.log(x);
}).catch(ex => {
    console.log(ex.toString());
});

The API specifications are not all too user-friendly, but I guess I can't ask for too much since this isn't an API designed to be used outside of Google's travel website.

It looks like the required parameters are mainly the f.req form data that needs to be submitted in the POST body. There are a lot of things going on in the single f.req parameter, but for now, I am concentrating on the core, which would be the date ranges and the start/destination airport codes. There does seem to be an option to filter out "non-stop flights only" as well.

Hotel API

Moving on, looking at hotel prices, it seems like there's a generic API called batchexecute which is used for various data queries. I can use it to search for hotels using a query string like "Las Vegas" or "Bellagio".

/**
 * Searches hotel using query string
 * @param searchQuery - the string to query hotels by
 */
async function searchHotels(searchQuery: string) {
    const hotelData = [[["AtySUc", JSON.stringify([searchQuery]), null, "1"]]];
    const queryData = encodeURIComponent(JSON.stringify(hotelData));
    const { data } = await axios({
        method: "POST",
        url: "    https://www.google.com/_/TravelFrontendUi/data/batchexecute",
        data: `f.req=${queryData}`,
        headers: { "Content-Type": "application/x-www-form-urlencoded;charset=UTF-8" },
    });
    return data;
}

/**
 * @param hotel_id - Hotel unique identifier, used by google travel website. This is returned by the hotel search API 
 * @param date_start - starting date as number array. Example : [2023, 6, 1]
 * @param date_end - ending date as number array. Example : [2023, 8, 1]
 */
async function getHotelPrices(hotel_id: string, date_start: number[], date_end: number[]) {
    const hotelData = [[["yY52ce", `[null,[${JSON.stringify(date_start)},${JSON.stringify(date_end)},1],[2,[],0],\"${hotel_id}\",\"USD\"]`, null, "generic"]]];
    const queryData = encodeURIComponent(JSON.stringify(hotelData));
    const { data } = await axios({
        method: "POST",
        url: "    https://www.google.com/_/TravelFrontendUi/data/batchexecute",
        data: `f.req=${queryData}`,
        headers: { "Content-Type": "application/x-www-form-urlencoded;charset=UTF-8" },
    });
    return data;
}


// -------------------//
// Test code
// -------------------//
seachHotels(
    "SFO hotels"
).then(x => {
    console.log(x);
}).catch(ex => {
    console.log(ex.toString());
});

getHotelPrices(
    "REPLACE_ME_WITH_HOTEL_ID",
    [2023, 6, 1],
    [2023, 8, 1]
).then(x => {
    console.log(x);
}).catch(ex => {
    console.log(ex.toString());
});

Development

After cleaning up the queried data (which has a lot of unneeded data), I was able to build the appropriate APIs needed for my application. And now, it's time to build an app!

Concept Sketch

Designed a simple application that can query and compare prices based on where you want to travel (from/to), how long you will be staying (x nights), and which hotel you are planning on staying at.

Libraries

For my frontend application, I am using the following libraries:

• React.js - the main frontend framework

• Vite.js - for the tooling

• Typescript - for types

• Jotai - for some state management

This was the first time trying out the Jotai library for state management in a React application. My app is small enough that it wasn't very complicated to manage the application state, but I decided to try out a new library for learning purposes. In the future, I'll try to write another article to explain my thoughts on it. For now, I have positive feelings about its simplicity.

Result

The finished source code is now uploaded to this repo:

https://github.com/icecreamsoft/travel-search-app

I've also deployed a demo application on AWS:

https://afuolvkiag.execute-api.us-east-1.amazonaws.com

After Thoughts

This was a fairly small demo project to see what is possible in the space of travel applications. Unsurprisingly, getting data like flight pricing or hotel pricing does not seem to be readily available through free APIs. I imagine with more reverse engineering we can clone the functionalities of sites like Google Flights, but this is not something that sounds too interesting. It's also not worth the trouble since Google could change their API specifications anytime and break your application anytime they want.

For now, I'll leave this experiment as is, and move on to the next project. Thanks for reading!