json - Pass Text from stdin as an Argument to a Shell Script from Python -


i have json file on have following:

  1. run "text" field on json through syntaxnet.
  2. from syntaxnet output, create new json field looks like: text_syntaxnet = [{'word' = <word1>, 'position = <word_position>, 'pos_tag' = <pos_tag>}, {...........}]
  3. add new json field original json came in input.

i doing using pig streaming. stream input data function parse.py contents are:

import sys import re import subprocess import json   def create_new_json_field(tags_list):     word_tags = {}     new_json_field = []     line in tags_list:         line = line.strip()         if not line:             continue         else:             words = line.split()             word_tags['word'] = words[1]             word_tags['position'] = words[0]             word_tags['pos_tag'] = words[4]             new_json_field.append(word_tags.copy())     return new_json_field   def main(argv):     try:         line in sys.stdin:             json_original = json.loads(line)             print json_original             tags = subprocess.check_output('./parse.sh %s' % line, shell=true)             tags_list = tags.split('\n')             new_json_field = create_new_json_field(tags_list)             result = json_original['text_syntaxnet'] = new_json_field             print new_json_field             print result     except exception e:         sys.stdout.write(str(e))  main(sys.argv) 

the contents of parse.sh are:

#!/bin/sh cd ........../models/syntaxnet jq --raw-output '.["text"]' | syntaxnet/demo.sh 

this code call parse.sh not work. rest works. not sure if syntax of command or environment issue. 1 please me debug problem.

note: subporcess call works when don't for line in sys.stdin in parse.py. want because want parse line line , create json objects.

thanks!


Comments

Popular posts from this blog

javascript - Thinglink image not visible until browser resize -

firebird - Error "invalid transaction handle (expecting explicit transaction start)" executing script from Delphi -

mongodb - How to keep track of users making Stripe Payments -