RH 95 metric

[rdaisatusq]

Hi everyone, I am following the python workflow found in this link (https://lpdaac.usgs.gov/resources/e-learning/getting-started-gedi-l2a-version-2-data-python/) to extract Rh metric to my study area.

I am on section "6.Spatial Visualization", particularly:

# Set up lists to store data
shotNum, dem, zElevation, zHigh, zLat, zLon, rh25, rh98, rh100 ,quality ,degrade, sensitivity ,beamI, selectedAlgorithm = ([] for i in range(14))


# Loop through each beam and open the SDS needed
for b in beamNames:
[shotNum.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/shot_number') and b in g][0]][()]]
[dem.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/digital_elevation_model') and b in g][0]][()]]
[zElevation.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/elev_lowestmode') and b in g][0]][()]]
[zHigh.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/elev_highestreturn') and b in g][0]][()]]
[zLat.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/lat_lowestmode') and b in g][0]][()]]
[zLon.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/lon_lowestmode') and b in g][0]][()]]
[rh25.append(h[25]) for h in gediL2A[[g for g in gediSDS if g.endswith('/rh') and b in g][0]][()]]
[rh98.append(h[98]) for h in gediL2A[[g for g in gediSDS if g.endswith('/rh') and b in g][0]][()]]
[rh100.append(h[100]) for h in gediL2A[[g for g in gediSDS if g.endswith('/rh') and b in g][0]][()]]
[quality.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/quality_flag') and b in g][0]][()]]
[degrade.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/degrade_flag') and b in g][0]][()]]
[sensitivity.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/sensitivity') and b in g][0]][()]]
[beamI.append(h) for h in [b] * len(gediL2A[[g for g in gediSDS if g.endswith('/shot_number') and b in g][0]][()])]
[selectedAlgorithm.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/selected_algorithm') and b in g][0]][()]]

I actually would like to add the relative metric (rh) 95. By adding the following rh95 in the codes (shown below):

# Set up lists to store data shotNum, dem, zElevation, zHigh, zLat, zLon, rh25, rh95, rh98, rh100 ,quality ,degrade, sensitivity ,beamI, selectedAlgorithm = ([] for i in range(15))

# Loop through each beam and open the SDS needed for b in beamNames:
[shotNum.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/shot_number') and b in g][0]][()]]
[dem.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/digital_elevation_model') and b in g][0]][()]]
[zElevation.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/elev_lowestmode') and b in g][0]][()]]
[zHigh.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/elev_highestreturn') and b in g][0]][()]]
[zLat.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/lat_lowestmode') and b in g][0]][()]]
[zLon.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/lon_lowestmode') and b in g][0]][()]]
[rh25.append(h[25]) for h in gediL2A[[g for g in gediSDS if g.endswith('/rh') and b in g][0]][()]]
   [rh95.append(h[95]) for h in gediL2A[[g for g in gediSDS if g.endswith('/rh') and b in g][0]][()]]
   [rh98.append(h[98]) for h in gediL2A[[g for g in gediSDS if g.endswith('/rh') and b in g][0]][()]]
[rh100.append(h[100]) for h in gediL2A[[g for g in gediSDS if g.endswith('/rh') and b in g][0]][()]]
[quality.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/quality_flag') and b in g][0]][()]]
[degrade.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/degrade_flag') and b in g][0]][()]]
[sensitivity.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/sensitivity') and b in g][0]][()]]
[beamI.append(h) for h in [b] * len(gediL2A[[g for g in gediSDS if g.endswith('/shot_number') and b in g][0]][()])]
[selectedAlgorithm.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/selected_algorithm') and b in g][0]][()]]

Unfortunately I am getting the below error:

---------------------------------------------------------------------------
ValueError Traceback (most recent call last)
~\AppData\Local\Temp\ipykernel_16576\2450686660.py in <module>
2 allDF = pd.DataFrame({'Shot Number': shotNum, 'Beam': beamI, 'Latitude': zLat, 'Longitude': zLon, 'Tandem-X DEM': dem,
3 'Elevation (m)': zElevation, 'Canopy Elevation (m)': zHigh, 'Canopy Height (rh100)': rh100, 'RH 98': rh98,
----> 4 'RH 25': rh25, 'RH 95': rh95, 'Quality Flag': quality, 'Degrade Flag': degrade, 'Sensitivity': sensitivity})

~\AppData\Local\anaconda3\envs\geditutorial\lib\site-packages\pandas\core\frame.py in __init__(self, data, index, columns, dtype, copy)
612 elif isinstance(data, dict):
613 # GH#38939 de facto copy defaults to False only in non-dict cases
--> 614 mgr = dict_to_mgr(data, index, columns, dtype=dtype, copy=copy, typ=manager)
615 elif isinstance(data, ma.MaskedArray):
616 import numpy.ma.mrecords as mrecords

~\AppData\Local\anaconda3\envs\geditutorial\lib\site-packages\pandas\core\internals\construction.py in dict_to_mgr(data, index, columns, dtype, typ, copy)
463
464 return arrays_to_mgr(
--> 465 arrays, data_names, index, columns, dtype=dtype, typ=typ, consolidate=copy
466 )
467

~\AppData\Local\anaconda3\envs\geditutorial\lib\site-packages\pandas\core\internals\construction.py in arrays_to_mgr(arrays, arr_names, index, columns, dtype, verify_integrity, typ, consolidate)
117 # figure out the index, if necessary
118 if index is None:
--> 119 index = _extract_index(arrays)
120 else:
121 index = ensure_index(index)

~\AppData\Local\anaconda3\envs\geditutorial\lib\site-packages\pandas\core\internals\construction.py in _extract_index(data)
633 lengths = list(set(raw_lengths))
634 if len(lengths) > 1:
--> 635 raise ValueError("All arrays must be of the same length")
636
637 if have_dicts:

ValueError: All arrays must be of the same length

Hopefully the helpful community could give me some directions to also extract RH 95 in my .h5 data.

Many thanks!

Cheers!

-Richard

[LP DAACx - dgolon]

hi @rdaisatusq Our developers are looking into this. We'll report back when we have more information. Thanks

[LP DAAC - mahsajami]

Your code looks good. make sure you restart your notebook. If you run any variables outside of a loop before setting it to an empty list, the outputs will be appended to the existing one. That is why when you want to create a dataframe they have different dimensions/lengths.
example of the error you are experiencing:
shotNum, rh95= ([] for i in range(2))
run the first loop once and the other one twice.
for b in beamNames:
[shotNum.append(h) for h in gediL2A[[g for g in gediSDS if g.endswith('/shot_number') and b in g][0]][()]]
for b in beamNames:
[rh95.append(h[95]) for h in gediL2A[[g for g in gediSDS if g.endswith('/rh') and b in g][0]][()]]

Now, if you run the code below to create the pandas dataframe, `rh95` and `shotNum`, it will fail because you are trying to make a table with columns of different length.

allDF = pd.DataFrame({
'Shot Number': shotNum,
'RH 95': rh95,
})

Please let us know if you are still having issue adding Rh95 to your list.

Thanks,
Mahsa