In this article, we’ll tell you about original data (Standard Data Product, High-level Product, etc.) from the satellite-data-provider and how to get it.

What is original data from the satellite-data-provider?

As a satellite data platform, Tellus offers satellite data in two main formats.

The first format is map tile (PNG images on Mercator’s projection).
Map tiles are suitable for map handling on the web.

The other is unprocessed data supplied from satellite data providers.
When you handle data of a wide geographic area at one moment in time or you need precise locations, this format is recommended.

The following three types of data are available as of February 2020.

• PALSAR-2 L1.1 (the CEOS format)
• PALSAR-2 L2.1 (GeoTIFF)
• AVNIR-2 Ortho Rectified Image Product (GeoTIFF)

The two types of data with a processing level called LX.X are both Standard Data Products created by JAXA.

The article below explains Standard Data Product and High-level Product in detail, including what L1.1 means.

We’ll continue to add scenes and data types.

In this article, we’ll explain PALSAR-2 L1.1, a special type of satellite data in the CEOS format.

What is the CEOS format?

The CEOS format is the data format defined by the Committee on Earth Observation Satellites, an organization set up to have coordination between space agencies from different countries as well as international institutions.

What binary data is put where depends on the satellite and sensor, so if you use observation data that’s not supported by existing software or libraries, you will need to read documents offered by the data provider.

PALSAR-2 Product Format Description (JAXA)

The CEOS format is supposed to mainly be used by scientists and professionals. It might be a bit hard to use for people who are not familiar with satellite data.

However, for those who are interested in what satellite-data-provider original data looks like, we’ll tell you in plain simple words how to get PALSAR-2 L2.1 (GeoTiFF) data on the development environment.

How to get data?

You can get the data file by following these three steps.

The API is only available on the development environment.

Do a file search
Do a file search to get metadata
↓
Get the download URLs
Get the file download URLs from the ID of the needed file.
↓
Download them
Download and save the files on the development environment.

Get PALSAR-2 L2.1 data

Now, let’s get data!

This time, we will get PALSAR-2 L2.1 data as an example.

See Tellusの開発環境でAPI引っ張ってみた (Use Tellus API from the development environment) for how to use the development environment (JupyterLab) on Tellus.

File search (PALSAR-2 L2.1)

https://file.tellusxdp.com/api/v1/origin/search/palsar2-l21

Arguments

All the arguments are optional. If you don’t specify arguments, you get unfiltered results.

The API returns arrays of metadata.

Metadata in a return value

Sample code

import requests
from pprint import pprint

TOKEN = 'ここには自分のアカウントのトークンを貼り付ける'

def search_file(params={}, next_url=''):
    if len(next_url) > 0:
        url = next_url
    else:
        url = 'https://file.tellusxdp.com/api/v1/origin/search/palsar2-l21'
        
    headers = {
        'Authorization': 'Bearer ' + TOKEN
    }

    r = requests.get(url, params=params, headers=headers)
    if not r.status_code == requests.codes.ok:
        r.raise_for_status()
    return r.json()

ret = search_file({'after': '2018-10-05T15:00:00'})

pprint(ret['count'])
pprint(ret['next'])
pprint(ret['items'])

Get the token from APIアクセス設定 (API access setting) in My Page (login required).

When you run the code, it will show you the output below.

We specified the argument to get data for October 15, 2018 or later, and got the corresponding data.

We couldn’t get all the corresponding results because there were more than 100 results (the default number of results returned is 100; you can change it to a maximum of 1000).

Use the URL returned in “next” for next_url to get the next results.

Get the download URLs (PALSAR-2 L2.1)

If you find the data you want, then get the download URLs for it.

Every time you send a request, it gives you random URLs. Be aware of the expiration time of those URLs.

If you find the data you want, then get the download URLs for it.

Every time you send a request, it gives you random URLs. Be aware of the expiration time of those URLs.

https://file.tellusxdp.com/api/v1/origin/publish/palsar2-l11/{dataset_id}

Assign the name of the needed dataset (the scene ID in the case of PALSAR-2) to dataset_id.

Return value

ISO8601

The download URL becomes invalid after this.

Sample code

def publish_files(dataset_id):
    url = 'https://file.tellusxdp.com/api/v1/origin/publish/palsar2-l21/{}'.format(dataset_id)
    
    headers = {
        'Authorization': 'Bearer ' + TOKEN
    }

    r = requests.get(url, headers=headers)
    if not r.status_code == requests.codes.ok:
        r.raise_for_status()
    return r.json()

published = publish_files('ALOS2217642720-180604')

pprint(published)

We made a request using the value ‘ALOS2217642720-180604’ for dataset_id as an example and got an output as shown above.

We got a download URL for each file that belongs to the Scene of the ID ALOS2217642720-180604.

Download them

Finally, download the files using the URLs you received.

This time, we downloaded the IMG-HH-ALOS2217642720-180604-UBSR2.1GUD.tif (the second file), which belongs to the requested scene ALOS2217642720-180604.

def download_file(url):
    headers = {
        'Authorization': 'Bearer ' + TOKEN
    }
    r = requests.get(url, headers=headers)
    if not r.status_code == requests.codes.ok:
        r.raise_for_status()
    return r.content

def save_file(data, name):
    with open(name, "wb") as f:
        f.write(data)
        
file = download_file(published['files'][1]['url'])
save_file(file, published['files'][1]['file_name'])
print('done')

If you’ve successfully downloaded the file, it displays “done” (it takes a while due to its large size).

Because PALSAR-2 L2.1 data is GeoTIFF, you can easily read and display it using Python.

Sample code

from osgeo import gdal, gdalconst, gdal_array
from skimage import io
%matplotlib inline

tif = gdal.Open('IMG-HH-ALOS2217642720-180604-UBSR2.1GUD.tif', gdalconst.GA_ReadOnly)


# ファイル情報
print(tif.GetProjection())
print(tif.GetGeoTransform())
print(tif.RasterXSize, tif.RasterYSize)
print(tif.GetMetadata())
print('バンド数: {}'.format(tif.RasterCount))

# 画像表示
# 画像のサイズが27640 * 31340ピクセルと大きいため、すべてを一度に表示することができません。
# サンプルでは部分的に表示しています。
io.imshow(tif.GetRasterBand(1).ReadAsArray()[10000:11000, 10000:11000])

In this article; how to get original data from the satellite-data-provider on the development environment of Tellus.

We plan to offer more original data from the satellite-data-provider in addition to PALSAR-2’s. We also plan to publish data analysis tools.

Check it out.

Reference

ALOS-2/PALSAR-2 Level 1.1/1.5/2.1/3.1 CEOS SAR Product Format Description