Ldiversity and missing values

Please specify
SDC tool used: (mu-argus, sdcMicro, sdcMicroGUI, tau-argus, sdcTable)
Version used:
Operating system used: (Windows, Linux)

I am using sdcMicro 5.7.1 on Windows.

When I use ldiversity function with a sensitive variable that has many missing values, I get indices 1 when the sensitive variable is missing. As I understand, if the sensitive variable is missing then there is no disclosure issue related to the sensitive variable and the ldiversity index should not be 1.

I wonder if I am missing something or the calculation is incorrect?

Thank you!

Marcos

@msanches35

hi @msanches35 ; could you please provide a quick reproducible example using ldiversity()? thx!

Here is a reproducible example, based on Table 4 from the sdcMicro paper.

library("sdcMicro")
example.data <- structure(
  list(
    key1 = structure(
      c(
        1L,1L,1L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,
        2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,3L,3L,3L,3L,3L,3L,3L,3L,
        3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,
        3L,3L,3L,3L,3L,3L,3L,3L,3L,3L,4L,4L,4L,4L,5L,NA
      ),
      .Label = c("Do not know", "Female", "Male",  "Other", "Prefer not to answer"),
      class = "factor"
    ),
    key2 = structure(
      c(
        1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,
        1L,1L,2L,2L,2L,2L,2L,2L,2L,2L,2L,2L,1L,1L,1L,1L,1L,1L,1L,1L,
        1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,1L,2L,2L,
        2L,2L,2L,2L,2L,2L,2L,2L,2L,NA,1L,1L,1L,1L,NA,2L
      ),
      .Label = c("Elective", "Emergency", "Urgent"),
      class = "factor"
    ),
    key3 = structure(
      c(
        NA,NA,NA,1L,1L,1L,2L,3L,4L,5L,5L,5L,NA,NA,NA,NA,NA,NA,NA,
        NA,NA,NA,1L,1L,2L,NA,NA,NA,NA,NA,NA,NA,1L,1L,1L,2L,5L,5L,
        5L,5L,5L,5L,5L,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,
        NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,2L,NA,NA,NA,NA,1L,NA
      ),
      .Label = c("CENTRAL", "CENTRAL EAST", "CENTRAL WEST", "ERIE ST. CLAIR", "TORONTO CENTRAL"),
      class = "factor"
    ),
    health = c(
      NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,5,NA,NA,4,3,NA,NA,10,NA,NA,NA,
      NA,1,8,NA,NA,11,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,
      NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,NA,7,9,NA,NA,NA,NA,NA,NA,
      NA,NA,NA,NA,NA,NA,NA,6,2,NA,NA,NA,12,NA
    ),
    id = c(
      4,42,59,20,24,54,7,72,73,16,21,68,9,25,28,36,38,43,57,67,71,
      76,30,75,33,12,14,32,34,41,45,47,26,39,74,17,15,18,22,60,62,
      64,69,2,3,8,13,19,27,35,37,48,51,52,53,55,61,66,10,29,31,40,
      44,46,49,50,56,63,65,11,1,6,58,70,23,5
    )
  ),
  class = "data.frame",
  row.names = c(NA,-76L)
)
  
sdc <- createSdcObj(example.data, keyVars = c("key1","key2","key3"))
res1 <- ldiversity(sdc,ldiv_index = "health", missing = NA)
example.data$l_diversity <- res1@risk$ldiversity[,1]
head(example.data,20)

Rows 1, 2, 3 have l-diveristy 1. However, no disclosure issue here, since the sensitive variable is missing. Rows 7, 8 and 9 should also not pose concern in terms of l-diversity.

This is creating a problem because I cannot tell based on the l-diveristy column where are the disclosure risks. At least for me, it would make more sense if whenever there is at least one missing in the key, the l-diversity would be NA.

Thanks!

@msanches35

@msanches35 thx a lot. i will have a look.

@msanches35

@msanches35 i agree that the implementation of l-diversity risk is somewhat ambiguous. i just tried the following as the underlying c++ code does not seem to work well with NAs; i just tried this (using example.data as starting-point):

example.data$health_int <- as.integer(example.data$health)
example.data$health_int[is.na(example.data$health_int)] <- as.integer(-1)
sdc_int <- createSdcObj(example.data, keyVars = c("key1", "key2", "key3"))
res_int <- ldiversity(sdc_int, ldiv_index = "health_int", missing = as.integer(-1))
example.data$l_diversity_int <- res_int@risk$ldiversity[, 1]
head(example.data, 20)

which gave

          key1     key2            key3 health id health_int l_diversity_int l_diversity
1  Do not know Elective            <NA>     NA  4         -1               0           1
2  Do not know Elective            <NA>     NA 42         -1               0           1
3  Do not know Elective            <NA>     NA 59         -1               0           1
4       Female Elective         CENTRAL     NA 20         -1               0           3
5       Female Elective         CENTRAL     NA 24         -1               0           3
6       Female Elective         CENTRAL     NA 54         -1               0           3
7       Female Elective    CENTRAL EAST     NA  7         -1               0           1
8       Female Elective    CENTRAL WEST     NA 72         -1               0           1
9       Female Elective  ERIE ST. CLAIR     NA 73         -1               0           1
10      Female Elective TORONTO CENTRAL     NA 16         -1               2           3
11      Female Elective TORONTO CENTRAL      5 21          5               2           3
12      Female Elective TORONTO CENTRAL     NA 68         -1               2           3
13      Female Elective            <NA>     NA  9         -1               0           1
14      Female Elective            <NA>      4 25          4               1           1
15      Female Elective            <NA>      3 28          3               1           1
16      Female Elective            <NA>     NA 36         -1               0           1
17      Female Elective            <NA>     NA 38         -1               0           1
18      Female Elective            <NA>     10 43         10               1           1
19      Female Elective            <NA>     NA 57         -1               0           1
20      Female Elective            <NA>     NA 67         -1               0           1

which are more plausible results; i guess we probably should provide a possibility to allow (automatic) recoding of missings in the ldiv_index so some (unused) integer-variable or at least add a message that using NA might lead to unexpected results.

Thanks, the results do make more sense, I think it will help me to get an useful ldiversity index! But there are still some inconsistencies in my view:

Rows 10:12 - Given that NA can be anything, I think index should be 3; or it should simply be 0 every time there is at least one NA in the sensitive variable.
Rows 14 and 15 and 18 should not be 1. However, I think this may be resolved if we simply recode the NA in key3 into some non-NA value.

Thanks again!

i have now done some investigation, but before comitting these I want to gather some feedback on "edge-cases"; the most-simple case (so missing values) is trivial and the expected results should be non-ombiguous.

> df
  key1 health ldiv
1    a      1    3
2    a      1    3
3    a      2    3
4    a      3    3
5    b      5    1
6    b      5    1

in the data.frame from above, ldiv should report values 3 (for key1 == a) and 1 (for key1 == b) given sensitive variable health; it is however not 100% clear (at least to me) what the expected results are if there a missing values in the sensitive-variable (given a complete key);

> df
  key1 health
1    a     NA
2    a     NA
3    a     1

the current implementation counts each NA (in the sensitive variable) as separate, unique value; therefore in the data.frame from above, the result of the ldiv-measure would be 3. If you however have a unique key (fk = 1 / a single row in the data.frame as shown below)

> df
  key1 health
1    a     NA

and the information in the sensitive-varible is missing, the result (using this logic) would be 1 even though there is obviously no disclosure problem (because the value in the sensitive-variable is completely missing). but any data.frame with such a key could not be e.g 2-diverse. in this case, i would have expected the result to be either the total number of characteristics in the sensitive variable over the entire data.set or even NA. on the other hand, it would also be perfectly fine to argue that since this key has a value of fk=1 (sample unique) it would violate 2-anonymity and therefore would also of course have to violate 2-diversity which would make the value of 1 for the ldiv-measure “explainable”.

another issue that currently leads to results that I was not really expecting is that for keys that are completely missing (NA in all key-variables), the ldiversity-measure yields a value of 0. in this case, I would have expected NA or even the total number of possible characteristics in the sensitive variable over the entire dataset but certainly not 0.

what are your thoughts on this?

We are back to the old problem about frequency estimation with missing values. Mustn't we argue with probabilities that a NA is of a certain category.
With assumptions that the attacker predictive power is zero to estimate NA's correctly, knowledge how many different categories exists (reasonable assumption) and that the frequencies are balanced and does not depend on other variables:

> df
  key1 health l-diversity
1    a     NA    2.111
2    a     NA   2.111
3    a     1       1

To see how to get to 2.111 is best to draw a decision tree. The probability of NA beging 1, 2 or 3 is 1/3 of each NA, so you can have with probability 1/9 the events 1,1,1 with outcome 1, then 2,1,1 --> 2, then 3,1,1 --> 2, then 1,2,1 --> 2, then 2,2,1 --> 2,then 3,2,1 --> 3, then 1,3,1 --> 2, then 2,3,1 --> 3, then 3,3,1 --> 2 thus 1+2+2+2+2+3+2+3+2 = 19 and l-div of 19/9 = 2.1111)

This seems manageable to calculate in general. But what if these assumptions does not hold? If the assumption on balanced frequencies does not hold, this is still manageable.
The probabilities based on observed frequencies are then --> assume they are 1/6, 2/6, 3/6 that NA is category 1, 2, or 3 in the example. Then we get
11/36 + 21/18 + 21/12 + 21/18 + 21/9 + 31/6 +2* 1/12 + 31/6 + 21/4 = 2.305

If the attacker can build a model with some good predictive power to estimate the NA's correctly then I feel lost to estimate the l-diversity under missing values.

What I wanted to express is that the estimation of l-diversity is only straightforward when restrictive assumptions are made. Your assumptions are even stronger than mine above.

Sorry about the delay on this. Here is what I thought, in case you are still looking into this.

I think I understand the point.

Maybe a practical approach. I like the calculation that lead to 2.111 in the example, it could be the default. And maybe an option for the user to set the l-div = NA when sensitive variable = NA (although that is easy to do in case one does not agree with the use of the equal frequency). Counting NA as different values could be another option for the user, the best case-scenario.

In the example above, I don't understand why l-div = 1 for the last line. It seems to me that this l-diversity depends on the NA. When an invader sees that key1 = a, they still dont know what is the value for "health", therefore a subject for whom key1 = a is not at risk of having their health value disclosed. So, it would be 2.111 by default, or 3 in the best case scenario, or NA if the users chooses to do so. The fact that l-div = 1 in that last line and I am not able to agree with it, is the only thing that is impeding me to use the 'ldiversity' function in my application right now.

Thank you!

Navigation Menu

Repository navigation

Ldiversity and missing values #189

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Ldiversity and missing values #189

Description

Activity

bernhard-da commented on Aug 16, 2022

msanches35 commented on Aug 17, 2022

bernhard-da commented on Aug 18, 2022

bernhard-da commented on Aug 18, 2022

msanches35 commented on Aug 22, 2022

bernhard-da commented on Sep 29, 2022

matthias-da commented on Sep 29, 2022

msanches35 commented on May 24, 2024

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions