Creating an Avro schema for a simple json

Vote count:

0

I'm trying to build an avro schema for the following json:

{
  "id":1234,
  "my_name_field": "my_name",
  "extra_data": {
      "my_long_value": 1234567890,
      "my_message_string": "Hello World!",
      "my_int_value":  777,
      "some_new_field": 1
  }
}

The value for 'id' and 'my_name_field' are known but the fields in 'extra_data' dynamically change and are unknown.

The avro schema I had in mind is:

{
    "name":"my_record",
    "type":"record",
    "fields":[
        {"name":"id", "type":"int", "default":0},
        {"name":"my_name_field", "type":"string", "default":"NoName"},
        { "name":"extra_data", "type":{"type":"map", "values":["null","int","long","string"]}     }        
    ]
}

My first idea was to make 'extra_data' a record with a map, but this does not work:

{ "name":"extra_data", "type":{"type":"map", "values":["null","int","long","string"]} }

I get:

AvroTypeException: Expected start-union. Got VALUE_NUMBER_INT

apache gives some nice examples in http://ift.tt/1dWAboS but none seem to do the job.

This is the unit test I run to check:

public class AvroTest {

@Test
public void readRecord() throws IOException {

    String event="{\"id\":1234,\"my_name_field\":\"my_name\",\"extra_data\":{\"my_long_value\":1234567890,\"my_message_string\":\"Hello World!\",\"my_int_value\":777,\"some_new_field\":1}}";

    SchemaRegistry<Schema> registry = new com.linkedin.camus.schema.MySchemaRegistry();
    DecoderFactory decoderFactory = DecoderFactory.get();

    ObjectMapper mapper = new ObjectMapper();
    GenericDatumReader<GenericData.Record> reader = new GenericDatumReader<GenericData.Record>();
    Schema schema = registry.getLatestSchemaByTopic("record_topic").getSchema();
    reader.setSchema(schema);

    HashMap hashMap = mapper.readValue(event, HashMap.class);
    long now = Long.valueOf(hashMap.get("now").toString())*1000;
    GenericData.Record read = reader.read(null, decoderFactory.jsonDecoder(schema, event));
}

Would appreciate help with this, Thanks.

asked 45 secs ago

Guy

77