Error in binarisaion of Grambank features missing values

[HedvigS]

I was at first a little confused at the -1 values in the Grambank sheet in this repos https://raw.githubusercontent.com/LeeLanguageLab/URIELPlus/refs/heads/main/urielplus/database/urielplus_csvs/grambank_data.csv". As far as I understand, -1 represents missing values and the reason why a NaN value wasn't chosen is to keep everything as int which has some benefits in NumPy world. Please correct me if I'm wrong, I'm more used to pandas when it comes to python.

I went through the features in your Grambank file and matched up to Grambank IDs, making this table Grambank_uriel+_matching.tsv. There isn't always a one-to-one correspondence, for example you have lumped a few cases that we split (reduplication, gender/number marking). For now, I didn't do the further calculation of 1 if at least 1 etc, I just left it at the straight comparisons where possible. (Coincidentally, this kind of lumping has some parallels to what we do in GBI logical (see #8). )

In your file , -1 does indeed reliably corresponds to ? values in Grambank v1.0.3.

Here's some R code to illustrate:

library(tidyverse)
#devtools::install_github("HedvigS/rgrambank", ref = "make_GBI-updates") #currently working out an issue with pipes, using non-main branch for now
library(rgrambank)

Grambank_ParameterTable <- utils::read.csv("https://raw.githubusercontent.com/grambank/grambank/refs/tags/v1.0.3/cldf/parameters.csv") |> 
  rgrambank::make_binary_ParameterTable() |> 
  dplyr::mutate(Grambank_ID_desc = stringr::str_replace(Grambank_ID_desc, " ", "_")) |> 
  dplyr::select(Parameter_ID = ID, Grambank_ID_desc)

Grambank_URIEL_match <- readr::read_tsv("Grambank_uriel+_matching.tsv", show_col_types = FALSE) |> 
  dplyr::full_join(Grambank_ParameterTable, by = "Grambank_ID_desc")

Grambank_ValueTable <- utils::read.csv("https://raw.githubusercontent.com/grambank/grambank/refs/tags/v1.0.3/cldf/values.csv") |> 
  rgrambank::make_binary_ValueTable()

Urielplus_grambank_sheet <- read.csv("https://raw.githubusercontent.com/LeeLanguageLab/URIELPlus/refs/tags/1.2/urielplus/database/urielplus_csvs/grambank_data.csv") |> 
  tidyr::pivot_longer(cols = -1) |> 
  dplyr::mutate(vale = ifelse(value == -1, NA, value)) |> 
  dplyr::select(Language_ID = code, "URIEL+_code" = name, "URIEL+_value" = value) 

matched <- Grambank_ValueTable |> 
  dplyr::select(Language_ID, Parameter_ID, Grambank_value = Value) |> 
  dplyr::left_join(Grambank_URIEL_match, by = "Parameter_ID") |> 
  dplyr::inner_join(Urielplus_grambank_sheet, by = join_by(Language_ID, `URIEL+_code`)) 

  dplyr::select(matched, Grambank_value, "URIEL+_value") |> table(useNA = "always")

which gives

              URIEL+_value
Grambank_value     -1      0      1   <NA>
          ?     75235   2720      0      0
          0         0 241234      0      0
          1         0      0 113921      0
          <NA>      0      0      0      0

However, sometimes a ? in Grambank is represented in the URIEL+ Grambank sheet as 0. Looking at these, they are all binarised versions of the multistate features.

>   matched |>
+     dplyr::filter(Grambank_value == "?" ) |> 
+     dplyr::filter(`URIEL+_value` == 0) |> 
+     dplyr::distinct("URIEL+_code", Grambank_ID_desc)
   "URIEL+_code"         Grambank_ID_desc
1    URIEL+_code      GB203b_UQOrder_N-UQ
2    URIEL+_code      GB203a_UQOrder_UQ-N
3    URIEL+_code    GB193b_ANMOrder_N-ANM
4    URIEL+_code    GB193a_ANMOrder_ANM-N
5    URIEL+_code   GB130b_IntransOrder_VS
6    URIEL+_code   GB130a_IntransOrder_SV
7    URIEL+_code GB065b_POSSOrder_PSD-PSR
8    URIEL+_code GB065a_POSSOrder_PSR-PSD
9    URIEL+_code    GB025b_DEMOrder_N-Dem
10   URIEL+_code    GB025a_DEMOrder_Dem-N
11   URIEL+_code    GB024b_NUMOrder_N-Num
12   URIEL+_code    GB024a_NUMOrder_Num-N

I used the rgrambank function make_binary_ValueTable to make Grambank v1.0.3 multistate features binary, which is based on the R script make_wide_binarized.R from the Grambank release paper (2023). If the value for a multistate feature is ?, then the derived binarised features are both ?. It seems that in the creation of your Grambank sheet in URIEL+, these data-points have instead become 0. I think this should be changed.

I am not sure if I've done this data processing right in terms of how URIEL+. Could you let me know how this looks to you?