json - Pass Text from stdin as an Argument to a Shell Script from Python -
i have json file on have following:
- run "text" field on json through syntaxnet.
- from syntaxnet output, create new json field looks like:
text_syntaxnet = [{'word' = <word1>, 'position = <word_position>, 'pos_tag' = <pos_tag>}, {...........}]
- add new json field original json came in input.
i doing using pig streaming. stream input data function parse.py
contents are:
import sys import re import subprocess import json def create_new_json_field(tags_list): word_tags = {} new_json_field = [] line in tags_list: line = line.strip() if not line: continue else: words = line.split() word_tags['word'] = words[1] word_tags['position'] = words[0] word_tags['pos_tag'] = words[4] new_json_field.append(word_tags.copy()) return new_json_field def main(argv): try: line in sys.stdin: json_original = json.loads(line) print json_original tags = subprocess.check_output('./parse.sh %s' % line, shell=true) tags_list = tags.split('\n') new_json_field = create_new_json_field(tags_list) result = json_original['text_syntaxnet'] = new_json_field print new_json_field print result except exception e: sys.stdout.write(str(e)) main(sys.argv)
the contents of parse.sh
are:
#!/bin/sh cd ........../models/syntaxnet jq --raw-output '.["text"]' | syntaxnet/demo.sh
this code call parse.sh not work. rest works. not sure if syntax of command or environment issue. 1 please me debug problem.
note: subporcess call works when don't for line in sys.stdin
in parse.py
. want because want parse line line , create json objects.
thanks!
Comments
Post a Comment