高手高高手请进：JAVA解析CSV文件-CSDN论坛
高手高高手请进：JAVA解析CSV文件
首先说明：本人已经用过正则，opencsv（apache的）,javacsv2.0（商业的）....&&try过无数次，都无法搞定！！
请不要告诉我用上述办法，甚至String.split(",")。。。这样最原始的搞法，因为我都试过
其实我的需求就一样事情：如何处理csv数据中包含的逗号？？如果不是这个原因我也不会发这个帖子了。。。
略略贴一贴正则的：
&&&&&&&&String&str&=&"7,,,,C.3,-,P,SYMBIAN,\"\"NOKIA,SAMSUNG,Sony&Erission\"\",9.0,NPS,\"\"N73,,E71,10C,N95\"\",";
&&&&&&&&String&regex&=&"\\G(?:^|,)(?:\"([^\"]*+(?:\"\"[^\"]*+)*+)\"|([^\",]*+))";
&&&&&&&&Matcher&main&=&Pattern.compile(regex).matcher(str);
&&&&&&&&Matcher&mquote&=&Pattern.compile("\"\"").matcher("");
&&&&&&&&while&(main.find())&{
&&&&&&&&&&&&String&
&&&&&&&&&&&&if&(main.start(2)&&=&0)&{
&&&&&&&&&&&&&&&&field&=&main.group(2);
&&&&&&&&&&&&}&else&{
&&&&&&&&&&&&&&&&field&=&mquote.reset(main.group(1)).replaceAll("\"");
&&&&&&&&&&&&}
&&&&&&&&&&&&System.out.println("Field&["&+&field&+&"]");
正常情况下,&str解析出来的东东应该是这样(数据用[]包含起来)：
[7],[],[],[],[C.3],[-],[P],[SYMBIAN],[NOKIA,SAMSUNG,SonyErission],[9.9],[NPS],[N73,,10C,N95]
但上述正则输出的不是这个内容，而且opencsv,javacsv2输出的也都不符合我的要求。其实主要就是红色部分不符合。
特发帖求解！！！
自己顶起！！
csv&&用excel打开&再导到数据库里
你应该修改你的源文件，修改成为符合标准CSV文件的格式：
CSV逗号分隔值文件
　　0&开头是不留空，以行为单位。
　　1&可含或不含列名，含列名则居文件第一行。
　　2&一行数据不垮行，无空行。
　　3&以半角符号，作分隔符，列为空也要表达其存在。
　　4&列内容如存在半角逗号（即,）则用半角引号（即""）将该字段值包含起来。
　　5&列内容如存在半角引号（即"）则应替换成半角双引号（""）转义。
　　6&文件读写时引号，逗号操作规则互逆。
　　7&内码格式不限，可为ASCII、Unicode或者其他。&
然后用标准CSV库解析。
比如XML，&xxx="&&"&没有那种解析引擎可以解析这样的东西
每天回帖即可获得10分可用分！小技巧：教您如何更快获得可用分&&
建议使用smooks这个插件。可以解析csv，xml，edi，flatfile等等。具体情况google一下吧。
像你这样的，是需求有问题，如果数据中有大量的逗号，干嘛还要用csv格式？
换成"|"或者tsv格式都可以，甚至xml格式
你这是自寻死路!
我觉得可以不用csv格式的，直接xls格式保存数据
然后用POI来操作excel
引用&6&楼&cooljia&的回复:像你这样的，是需求有问题，如果数据中有大量的逗号，干嘛还要用csv格式？
换成"|"或者tsv格式都可以，甚至xml格式
你这是自寻死路!
犀利哥吗？言辞好犀利，嘎嘎。。。
帅哥，问题是这个CSV并非我生成的，而是其他系统导出的，是不是要推动别人去修改它的代码啊。。。CSV生成很容易，解析就难了，就像MD5.。。
不过该问题已经曲线解决了。如同楼上有人说的，另存为XLS&+&POI...
而且感觉该问题肯定有解，否则EXCEL怎么能正确打开CSV。。。
String&str&=&"7,,,,C.3,-,P,SYMBIAN,\"\"NOKIA,SAMSUNG,Sony&Erission\"\",9.0,NPS,\"\"N73,,E71,10C,N95\"\",";
&&&&&&&&String&regex&=&"([^,]*)(,)";
&&&&&&&&Matcher&main&=&Pattern.compile(regex).matcher(str);
&&&&&&&&while&(main.find())&{
&&&&&&&&&&&&String&
&&&&&&&&&&&&field&=&main.group(1).replaceAll("\"","");
&&&&&&&&&&&&System.out.println("Field&["&+&field&+&"]");
Field&[C.3]
Field&[SYMBIAN]
Field&[NOKIA]
Field&[SAMSUNG]
Field&[Sony&Erission]
Field&[9.0]
Field&[NPS]
Field&[N73]
Field&[8250]
Field&[N97]
Field&[E61]
Field&[E71]
Field&[5320]
Field&[8855]
Field&[6210C]
Field&[N95]
引用&8&楼&whut_lcy&的回复:不过该问题已经曲线解决了。如同楼上有人说的，另存为XLS&+&POI...
如果楼主是手工把cvs另存为xls,那问题似乎还是没有完美解决。
引用&11&楼&mybeautiful&的回复:引用&8&楼&whut_lcy&的回复:
不过该问题已经曲线解决了。如同楼上有人说的，另存为XLS&+&POI...
如果楼主是手工把cvs另存为xls,那问题似乎还是没有完美解决。
是啊，明天验证下10楼的，如果解决，全额送分！
如果字段中含有逗号，那么这个字段用引号包起来，如果字段中含有引号，那就使用两个引号进行转义。
你示例的&CSV&中，有两个引号的，为什么没有看“应该”当中有引号啊？
10楼&的代码没贴全
引用&3&楼&dollyn&的回复:你应该修改你的源文件，修改成为符合标准CSV文件的格式：
CSV逗号分隔值文件
　　0&开头是不留空，以行为单位。
　　1&可含或不含列名，含列名则居文件第一行。
　　2&一行数据不垮行，无空行。
　　3&以半角符号，作分隔符，列为空也要表达其存在。
　　4&列内容如存在半角逗号（即,）则用半角引号（即""）将该字段值包含起来。
　　5&列内容如……
这才是正道
import&java.util.regex.M
import&java.util.regex.P
public&class&Test&{
&*&@param&args
public&static&void&main(String[]&args)&{
String&str&=&"7,,,,C.3,-,P,SYMBIAN,\"\"NOKIA,SAMSUNG,Sony&Erission\"\",9.0,NPS,\"\"N73,,E71,10C,N95\"\",";
&&&&&&&&String&regex&=&"([^,]*)(,)";
&&&&&&&&Matcher&main&=&Pattern.compile(regex).matcher(str);
&&&&&&&&while&(main.find())&{
&&&&&&&&&&&&String&
&&&&&&&&&&&&field&=&main.group(1).replaceAll("\"","");
&&&&&&&&&&&&System.out.println("Field&["&+&field&+&"]");
高深的技术哦&小弟受教育了
10楼的输出还是不对
Field&[C.3]
Field&[SYMBIAN]
Field&[NOKIA]
Field&[SAMSUNG]
Field&[Sony&Erission]
Field&[9.0]
Field&[NPS]
Field&[N73]
Field&[8250]
Field&[N97]
Field&[E61]
Field&[E71]
Field&[5320]
Field&[8855]
Field&[6210C]
Field&[N95]
N73,,E71,10C,N95&没在一行上
算了不等了，下班结贴。
3楼说的要规范CSV输出，这个不是我说了算。这个格式的CSV是另一个系统做的，与其推动其修改CSV，不如直接让它导出XLS。POI直接解析。
无论你什么样的CSV，只要告诉MS的EXCEL它的分隔符是什么，他都可以正确解析。纳了闷了。。。
另外，据说ms&的sql&server可以直接用sql语句将xls文件导入数据库。其他数据库应该不支持吧？毕竟xls是ms&的东西
学习下，不知结贴了&，发帖还能得分不
没有解决呀&失望&&
10楼的regex改成
String&regex&=&"((\"[^\"]*(\"{2})*[^\"]*\")*[^,]*)(,)";
就可以了。lz可以试一试
回复51CTO旗下网站
Smooks 1.2框架:处理XML与非XML的Java框架
Smooks 1.2框架是一个用于处理XML与非XML(CSV，EDI，Java等)的Java框架与引擎。它常常被认为是一个转换框架并以此被用于好几个产品和项目中，包括JBoss ESB(以及其它ESB)。
作者：佚名来源：ITPUB| 16:32
Smooks 1.2框架能够用于各种数据格式的转换：XML to XML，CSV to XML，EDI to XML，XML to EDI，XML to CSV，Java to XML，Java to EDI，Java to CSV，Java to Java，XML to Java，EDI to Java等。
不久前，Smooks 1.2框架版本发布了，这次添加了许多有趣的新功能：
EDIFACT Java 编译器(EJC)：EJC大大的简化了绑定EDI数据到Java object graph，EJC类似于JAXBs XJC。
支持实体持久性框架：重利用你的实体持久性资源(如Hibernate, iBATIS 火任何JPA兼容的资源)坚持和丰富任何格式的messages。访问一个数据库，使用它的查询语言，或CRUD方法 ，来读出/写入数据库。
验证：支持Data Field level验证，使用正则表达式，或Business Rule compliance验证，使用MVEL表达式。
简易的CSV处理：使用简易的XML配置，在几分钟之内即可绑定CVS记录到java 对象，或者使用一个更简单的programmatic API。
改进Programmatic API：Smooks 1.2之前的版本，programmatic配置不是Smooks的强项，但是在1.2版本中已经取得了重大的改进。许多Smooks的功能都可以通过java被利用，不需要编写XML。
使用Smooks 1.2框架的方式有两种，你可以使用其中之一或结合使用它们：
模式一：你可以完全投入到Smooks中，编写你自己的定制访问者逻辑事件处理器，将其用于处理一个数据源事件流中特定事件。使用这一模式，你必须熟悉核心的API。
模式二：你可以重用由Smooks发行版提供的开箱即用解决方案，其数目正在不断的增长中。在这种模式下，你只需要重用别人创建的组件即可，重新配置它们来处理你的数据源，例如，通过配置一些参数就可以由EDI数据源生成Java对象模型。【责任编辑： TEL：（010）】
大家都在看猜你喜欢
头条热点头条关注头条
24H热文一周话题本月最赞
讲师：742300人学习过
讲师：252235人学习过
讲师：13038人学习过
精选博文论坛热帖下载排行
本书是在《网管员必读―网络安全》第1版的基础上修改而成的。新版在保留第1版实用内容的基础上增加了大量新的实用内容，同时删除了一些过时...
订阅51CTO邮刊提供企业信息化整体解决方案
Smooks 1.2框架新功能介绍
Smooks 1.2框架新功能介绍
| 作者：慧都控件网 |
11:12:25| 阅读 0次
概述：不久前，Smooks 1.2框架版本发布了，这次添加了许多有趣的新功能：使用EDIFACT Java 编译器(EJC)，支持实体持久性框架，支持Data Field level验证等。
　　Smooks 1.2框架能够用于各种数据格式的转换：XML to XML，CSV to XML，EDI to XML，XML to EDI，XML to CSV，Java to XML，Java to EDI，Java to CSV，Java to Java，XML to Java，EDI to Java等。
　　不久前，Smooks 1.2框架版本发布了，这次添加了许多有趣的新功能：
　　EDIFACT Java 编译器(EJC)：EJC大大的简化了绑定EDI数据到Java object graph，EJC类似于JAXBs XJC.
　　支持实体持久性框架：重利用你的实体持久性资源(如Hibernate， iBATIS 火任何JPA兼容的资源)坚持和丰富任何格式的messages.访问一个数据库，使用它的查询语言，或CRUD方法 ，来读出/写入数据库。
　　验证：支持Data Field level验证，使用正则表达式，或Business Rule compliance验证，使用MVEL表达式。
　　简易的CSV处理：使用简易的XML配置，在几分钟之内即可绑定CVS记录到java 对象，或者使用一个更简单的programmatic API.
　　改进Programmatic API：Smooks 1.2之前的版本，programmatic配置不是Smooks的强项，但是在1.2版本中已经取得了重大的改进。许多Smooks的功能都可以通过java被利用，不需要编写XML.
　　使用Smooks 1.2框架的方式有两种，你可以使用其中之一或结合使用它们：
　　模式一：你可以完全投入到Smooks中，编写你自己的定制访问者逻辑事件处理器，将其用于处理一个数据源事件流中特定事件。使用这一模式，你必须熟悉核心的API.
　　模式二：你可以重用由Smooks发行版提供的开箱即用解决方案，其数目正在不断的增长中。在这种模式下，你只需要重用别人创建的组件即可，重新配置它们来处理你的数据源，例如，通过配置一些参数就可以由EDI数据源生成Java对象模型。
慧都控件｜提供软件技术整体解决方案
云集全球三千余款优秀控件、软件产品，提供行业领先的咨询、培训与开发服务
企业QQ：｜电话：023-
服务与支持
重庆 / 023-
北京 / 010-
400-700-1020
慧都科技有限公司 版权所有 Copyright 2003-
400-700-1020
<input type="hidden" id="url" value=""/>原文 http://www.smooks.org/mediawiki/index.php?title=V1.5:Smooks_v1.5_User_Guide
Smooks is an extensible framework for building applications for processing XML and non XML data (CSV, EDI, Java etc) using Java.
While Smooks can be used as a lightweight platform on which to build your own custom processing logic for a wide range of data formats, "out of the box" it comes with some very useful features that can be used individually, or seamlessly combined together:
Java Binding: Populate a Java Object Model from a data source (CSV, EDI, XML, Java etc). Populated object models can be used as a transformation result itself, or can be used by (e.g.) Templating resources for generating XML or other character based results. Also supports Virtual Object Models (Maps and Lists of typed data), which can be used by EL and Templating functionality.
Transformation: Perform a wide range of Data Transforms - XML to XML, CSV to XML, EDI to XML, XML to EDI, XML to CSV, Java to XML, Java to EDI, Java to CSV, Java to Java, XML to Java, EDI to Java etc.
Huge Message Processing: Process huge messages (GBs) - Split, Transform and Route message fragments to JMS, File, Database etc destinations.
Message Enrichment: Enrich a message with data from a Database, or other Datasources.
Complex Message Validation: Rules based fragment validation.
ORM Based Message Persistence: Use an entity persistence framework (like Ibatis, Hibernate or any JPA compatible framework) to access a database and use it's query language or CRUD methods to read from it or write to it. Use custom Data Access Objects (DAO's) to access a database and use it's CRUD methods to read from it or write to it.
Combine: Perform Extract Transform Load (ETL) operations by leveraging Smooks' Transformation, Routing and Persistence functionality.
As well as a ton of bug fixes, Smooks v1.5 includes the following new highlight features:
[] Simplify EDI Directories lookup. (By:&Renat Zubairov)
[] Namespace aware EDI and UN/EDIFACT Readers i.e. generation of namespaced event streams. (By:&Renat Zubairov)
[] XML Schema generation in ECT. The generated XSDs can validate the namespaced XML event stream generated by the UNEdifactReader (see MILYN-570 above) (By:&Renat Zubairov)
Java Binding
[] () Support for XML Reading and Writing through a new XMLBinding class. Prior to Smooks v1.5, the &jb:bean& binding configurations only allowed you to read data into a Java Object model. Serializing that Java Object model back to XML (or EDI, CSV etc) required you to write a template (e.g. FreeMarker) and configure that into Smooks. The XMLBinding class lets you both read and write/serialize XML (from Java objects) using only the standard &jb:bean& binding configurations i.e. no need for templates. (By:&Tom Fennelly)
Flat File Data Support
[] Simplify the process of implementing a Reader for Flat File data formats (flat record based formats). This includes support for simple Java binding configurations directly on the reader (SINGLE, LIST, MAP). Also includes support for "variable field records". The CSV reader now uses this API. (By:&Tom Fennelly)
[] Create a Regex based Flat File reader. Allows you to process any flat file structure, parsing the record fields using a regular expression. (By:&Tom Fennelly)
Camel/ServiceMix/OSGi Support
Various bug fixes and enhancements. (By:&Daniel Bevenius&and&Sorin Silaghi)
Project Housekeeping
[] Move from Maven to Gradle build. (By:&Daniel Bevenius)
[] Move to Nexus repository for artifacts. (By:&Tom Fennelly)
The easiest way to get started with Smooks is to download and try out some of the&. The examples are the perfect base upon which to integrate Smooks into your application.
For details on how to integrate Smooks into your project via Maven, see the&.
For details on how to integrate Smooks into your project via Ant, see the&.
Smooks can be download from the&&page.
The most commonly accepted definition of Smooks would be that it is a "Transformation Engine". However, at it's core, Smooks makes no mention of "data transformation". The&smooks-core&codebase is designed simply to support hooking of custom "" logic into an Event Stream produced by a data Source of some kind (XML, CSV, EDI, Java etc). As such,&smooks-core&is simply a "Structured Data Event Stream Processor".
Of course, the most common application of this will be in the creation of&Transformation&solutions i.e. implementing&&logic that uses the Event Stream produced from a Source message to produce a Result of some other kind. The capabilities in&smooks-core&enable more than this however. We have implemented a range of other solutions based on this processing model:
Java Binding: Population of a Java Object Model from the Source message.
Message Splitting & Routing: The ability to perform complex splitting and routing operations on the Source message, including routing to multiple destinations concurrently, as well as routing different data formats concurrently (XML, EDI, CSV, Java etc).
Huge Message Processing: The ability to declaratively consume (transform, or split and route) huge message without writing lots of high maintenance code.
As stated above, the basic principal of Smooks is to take a data&Source&of some kind (e.g. XML) and from it generate an&Event Stream, to which you apply&&logic&to produce a&Result&of some other kind (e.g. EDI).
Many different data Source and Result types are supported, meaning many different transformation types are supported, including (but not limited to):
XML to XML
XML to Java
Java to XML
Java to Java
EDI to XML
EDI to Java
Java to EDI
CSV to XML
CSV to ...
In terms of the Event Model used to map between the Source and Result, Smooks currently supports DOM and SAX Event Models. We will concentrate on the SAX event model here. If you want low level details on either models, please consult the&. The SAX event model is based on the hierarchical SAX events generated from an XML Source (startElement, endElement etc). However, this event model can be just as easily applied to other structured/hierarchical data Sources (EDI, CSV, Java etc).
The most important events (typically) are the&visitBefore&and&visitAfter&events. The following illustration tries to convey the hierarchical nature of these events.
In order to consume the SAX Event Stream produced from the Source message, you need to implement one or more of the&&interfaces (depending on which events you need to consume).
The following is a very simple example of how you implement&&logic and target that logic at the&visitBefore&and&visitAfter&events for a specific element in the Event Stream. In this case we target the&&logic at the &xxx& element events.
As you can see, the&&implement one method implementation per event. To target this implementation at the &xxx& element&visitBefore&and&visitAfter&events, we need to create a Smook configuration as shown (more on "Resource Configurations" in the following sections).
The Smooks code to execute this is very simple:
Smooks smooks = new Smooks("/smooks/echo-example.xml");&smooks.filterSource(new StreamSource(inputStream));
Note that in this case we don't produce a Result. Also note that we don't interact with the "execution" of the filtering process in any way, since we don't explicitly create an&&and supply it to the&.filterSource method call.
This example illustrated the lower level mechanics of the Smooks Programming Model. In reality however, users are not going to want to solve their problems by implementing lots Java code themselves from scratch. For this reason, Smooks is shipped with quite a lot of pre-built functionality i.e. ready to use&&logic. We bundle this&&logic based on functionality and we call the bundles "Cartridges".
Smooks executes by taking a data stream of one form or another (XML, EDI, Java, JSON, CSV etc) and from it, it generates an event stream, which it uses to fire different types of "&logic" (Java, Groovy, FreeMarker, XSLT etc). The result of this process can be to produce a new data stream in a different format (i.e. a traditional "transformation"), bind data from the source message data stream to java objects to produce a populated Java object graph (i.e. a "Java binding"), produce many smaller messages (message splitting) etc.
At a core level (inside Smooks), it just sees all of the "&logic" etc as "Smooks Resources" (SmooksResourceConfiguration) that are configured to be applied based on an event&selector&(i.e. event from the source data event stream). This is a very generic processing model and makes a lot of sense from the point of view of Smooks Core and it's architecture (maintainance etc). However, it can be a little too generic from a usability perspective because everything looks very similar in the configuration. To help with this, Smooks v1.1 introduced an "Extensible Configuration Model" feature. This allows specific resource types (Javabean binding configs, FreeMarker template configs etc) to be specified in the configuration using dedicated XSD namespaces of their own.
Example (Java Binding Resource):
&jb:bean beanId="lineOrder" class="example.trgmodel.LineOrder" createOnElement="example.srcmodel.Order"&
&jb:wiring property="lineItems" beanIdRef="lineItems" /&
&jb:value property="customerId" data="header/customerNumber" /&
&jb:value property="customerName" data="header/customerName" /&
&/jb:bean&
Example (FreeMarker Template Resource):
&ftl:freemarker applyOnElement="order-item"&
&ftl:template&&!-- &item&
&id&${.vars["order-item"].@id}&/id&
&productId&${.vars["order-item"].product}&/productId&
&quantity&${.vars["order-item"].quantity}&/quantity&
&price&${.vars["order-item"].price}&/price&
&/ftl:template&
&/ftl:freemarker&
When comparing the above examples to the pre Smooks v1.1 equivalents you can see that:
The user now has a more strongly typed configuration that is domain specific in each case (and so easier to read).
Because the v1.1+ configurations are XSD based, the user also gets autocompletion support from their IDE.
No longer any need to define the actual handler for the given resource type e.g. the BeanPopulator for java bindings.
Smooks Resource "selectors" are a very important part of Smooks and how it works. They instruct Smooks as to which message fragments to apply configured&&logic to, as well working as a simple opaque lookup value for non&&logic.
When the resource is a&&implementation (e.g. &jb:bean&, &ftl:freemarker& etc), Smooks will interpret the selector as an&&(like) selector. There are a number of things to be aware of:
The order in which the&&expression is applied is the reverse of normal order e.g. as applied by an XSLT. Smooks works backwards from the targeted fragment element, as opposed to forwards from the message root element.
Not all of the&&specification is supported. Selector supports the following XPath syntax:
'text()' and attribute (e.g. '@x') value selectors, with both Literal and Numeric e.g. "a/b[text() = 'abc']", "a/b[text() = 123]", "a/b[@id = 'abc']", "a/b[@id = 123]".
Note: 'text()' is only supported on the last selector step in an expression e.g. "a/b[text() = 'abc']" is legal while "a/b[text() = 'abc']/c" is illegal.
Note: 'text()' is only supported on SAXVisitor implementations that implement the SAXVisitAfter interface&only. If the SAXVisitor implements the SAXVisitBefore or SAXVisitChildren interfaces, an error will result.
And and Or logical operations e.g. "a/b[text() = 'abc' and @id = 123]", "a/b[text() = 'abc' or @id = 123]"
Namespaces on both the elements and attributes e.g. "a:order/b:address[@b:city = 'NY']".
Note: This requires the namespace prefix-to-URI mappings to be defined. If not defined, a configuration error will result. See the&&section for more details.
Supports "=" (equals), "!=" (not equals), "&" (less than), "&" (greater than).
Index selectors e.g. "a/b[3]"
Namespace prefix-to-URI mappings are configured through the core configuration namespace. These configurations are then available when resolving namespaced selectors.
&?xml version="1.0"?&
&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.4.xsd"&
&core:namespaces&
&core:namespace prefix="a" uri="http://a"/&
&core:namespace prefix="b" uri="http://b"/&
&core:namespace prefix="c" uri="http://c"/&
&core:namespace prefix="d" uri="http://d"/&
&/core:namespaces&
&resource-config selector="c:item[@c:code = '8655']/d:units[text() = 1]"&
&resource&com.acme.visitors.MyCustomVisitorImpl&/resource&
&/resource-config&
&/smooks-resource-list&
The basic functionality of Smooks Core can be extended through the creation of what we call a "Smooks Cartridge". A Cartridge is simply a Java archive (jar) containing reusable Content Handlers (Logic). A Smooks Cartridge should provide "ready to use" support for a specific type of XML analysis or transformation.
For a full list of the Cartridges supported by Smooks, see the&.
This is done by Smooks based on the following criteria:
If all&&resources* implement only the DOM&&interfaces (DOMElementVisitor or SerializationUnit), then the DOM processing model is selected.
If all&&resources* implement only the SAX&&interface (SAXElementVisitor), then the SAX processing model is selected.
If all&&resources* implement both the DOM and SAX&&interfaces, then the SAX processing model is selected, unless the Smooks resource configuration contains the &core:filterSettings type="DOM" /& (see below).
*&not including non element&&resources, like readers for example.
The following is an example of setting the filter type to DOM (SAX is the default):
&?xml version="1.0"?&
&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.4.xsd"&
&core:filterSettings type="DOM" /&
&/smooks-resource-list&
More information on global filtering settings can be found in the&.
The DOM processing model has the obvious:
Advantage&of being easier to work with on a code level, allowing node traversal etc. It also makes it a lot easier to take advantage of Scripting and Templating engines that have built in support for utilizing DOM structures (e.g.&&and&).
Disadvantage&of being constrained by memory i.e. if you have huge messages, then you typically cannot use a DOM processing model.
Smooks v1.1 added support for mixing these 2 models through the&DomModelCreator&class. When used with SAX filtering, this&&will construct a DOM Fragment of the visited element. This allows DOM utilities to be used in a Streaming environment.
When 1+ models are nested inside each other, outer models will never contain data from the inner models i.e. the same fragments will never coexist inside two models.
Take the following message as an example:
&order id="332"&
&customer number="123"&Joe&/customer&
&order-items&
&order-item id='1'&
&product&1&/product&
&quantity&2&/quantity&
&price&8.80&/price&
&/order-item&
&order-item id='2'&
&product&2&/product&
&quantity&2&/quantity&
&price&8.80&/price&
&/order-item&
&order-item id='3'&
&product&3&/product&
&quantity&2&/quantity&
&price&8.80&/price&
&/order-item&
&/order-items&
The DomModelCreator can be configured in Smooks to create models for the "order" and "order-item" message fragments:
&resource-config selector="order,order-item"&
&resource&org.milyn.delivery.DomModelCreator&/resource&&/resource-config&
In this case, the "order" model will never contain "order-item" model data (order-item elements are nested inside the order element). The in memory model for the "order" will simply be:
&order id='332'&
&customer number="123"&Joe&/customer&
&order-items /&
Added to this is the fact that there will only ever be 0 or 1 "order-item" models in memory at any given time, with each new "order-item" model overwriting the previous "order-item" model.&All this ensures that the memory footprint is kept to a minimum.
Because the Smooks processing model is event driven via the message content (i.e. you can hook in&&logic to be applied at different points while Smooks filters/streams the message), you can take advantage of this mixed DOM and SAX processing model.
See the following examples that utilize this mixed DOM + SAX approach:
The&bean context&(also known as "bean map") is a container for Objects which can be accessed within the Smooks filter process. One bean context is created per execution context (i.e. per Smooks.filterSource operation). Every bean, created by the cartridge, is put into this context under its&beanId. If you want the contents of the bean context to be returned at the end of the Smooks.filterSource process, supply a&org.milyn.delivery.java.JavaResult&object in the call to Smooks.filterSource method. The following example illustrates this principal:
//Get the data to filter
StreamSource source = new StreamSource(getClass().getResourceAsStream("data.xml"));
//Create a Smooks instance (cachable)
Smooks smooks = new Smooks("smooks-config.xml");
//Create the JavaResult, which will contain the filter result after filtering
JavaResult result = new JavaResult();
//Filter the data from the source, putting the result into the JavaResult
smooks.filterSource(source, result);
//Getting the Order bean which was created by the JavaBean cartridge
Order order = (Order)result.getBean("order");
If you need to access the bean context beans at runtime (e.g. from a customer&&implementation), you do so via the&BeanContext&object. The "BeanContext" object can be retrieved from theExecutionContext&via the&getBeanContext()&method. When adding or retrieving objects from the BeanContext you should first retrieve a&BeanId&object from the&BeanIdStore. The BeanId object is a special key that ensures higher performance then String keys, however String keys are also supported. The BeanIdStore must be retrieved from the&ApplicationContext&via the&getBeanIdStore()method. A BeanId object can be created by calling the&register("beanId name")&method. If you know that the BeanId is already registered then you can retrieve it by calling the&getBeanId("beanId name")&method. BeanId objects are ApplicationContext scoped objects. You normally register them in the initialization method of your custom visitor implemention and then put them as properties in the visitor object. You can then use them in the 'visitBefore' and 'visitAfter' methods. The BeanId objects and the BeanIdStore are thread safe.
A number of pre-installed beans are available in the Bean Context at runtime:
PUUID: UniqueId bean. This bean provides unique identifiers for the filtering ExecutionContext.&.
PTIME: Time bean. This bean provides time based data for the filtering ExecutionContext.&.
The following are examples of how each of these would be used in a FreeMarker template.
Unique ID of the ExecutionContext (message being filtered):
${PUUID.execContext}
Random Unique ID:
${PUUID.random}
Message Filtering start time (in milliseconds):
${PTIME.startMillis}
Message Filtering start time (in nanoseconds):
${PTIME.startNanos}
Message Filtering start time (Date):
${PTIME.startDate}
Time now (in milliseconds):
${PTIME.nowMillis}
Time now (in nanoSeconds):
${PTIME.nowNanos}
Time now (Date):
${PTIME.nowDate}
This section looks at the different ways in which Smooks can produce "Output" from the Filtering process.
Smooks can "present" output to the outside world in the following ways:
"In-Result" Instances: Returned in the&&instances passed to the&&method.
During the Filtering Process: Output generated and sent to external endpoints (ESB Services, Files, JMS Destinations, DBs etc) during the Filtering process. This is where message fragment events are used to trigger routing of message fragments to external endpoints e.g. when&&fragments of a message.
A very important point to remember is that Smooks can generate output/results in either or both of the above ways, all in a single filtering pass of a message stream. It doesn't need to filter a message stream multiple times in order to generate multiple outputs/results. This is critical in terms of performance/efficiency.
A look at the Smooks API reveals that Smooks can be supplied with multiple&&instances:
public void filterSource(Source source, Result... results) throws SmooksException
In terms of the types of&&that Smooks can work with, we're talking about the standard JDK&&and&&types, as well as some Smooks "specializations":
: Result type for capturing the contents of the Smooks Java Bean context.
: Result type for capturing&&Results.
: Simple Result type used mainly when writing tests. Simple&&extension wrapping a&.
This is obviously the most common method of capturing output from the Smooks filtering process.
As yet, Smooks does not support capturing of&&data to multiple&&instances of the same type. For example, you can specify multiple&&instances in the&method call, but Smooks will only output to one of these&&instances (the first one).
These&&types receive "special" attention from Smooks. As Smooks process a message&, it produces a stream of events. If a&&or&&is supplied in the&call, Smooks will (by default - see&&global parameter) serialize the event stream (produced from the&) to the supplied&&or&as XML. Obviously,&&logic can be configured/applied to the event stream before serialization.
This is the mechanism used to perform a standard 1-input/1-xml-output character based transformation.
Smooks is also able to generate different types of output during the&&process i.e. as it is filtering the message event stream and before it reaches the end of the message. A classic example of this being when it is used to&&message fragments to different types of endpoints for processing by other processes.
So one might wonder why Smooks doesn't "batch up" the message data and produce all the results/outputs after filtering the complete message. Well the answer is straightforward enough:
Performance!!
It's just easier this way because you can utilize the message event stream to trigger the fragment transform and routing operations.
Consider an Order message that has hundreds of thousands (or millions) of Order Items that need to be split out and routed to different departments in different formats, based on different criteria. The only way of handing messages of this magnitude is by streaming the process.
As Smooks performs the filtering process (processing the Event Stream generated from the Source), it publishes events that can be captured and programmatically analyzed during/after execution.? The easiest way to generate an execution report out of Smooks is to configure the ExecutionContext to generate a report. Smooks supports generation of a HTML report via the HtmlReportGenerator class.
The following is an example of how to configure Smooks to generate a HTML report.
Smooks smooks = new Smooks("/smooks/smooks-transform-x.xml");ExecutionContext execContext = smooks.createExecutionContext();&execContext.setEventListener(new HtmlReportGenerator("/tmp/smooks-report.html"));smooks.filterSource(execContext, new StreamSource(inputStream), new StreamResult(outputStream));
The HtmlReportGenerator is a very useful tool during development with Smooks. It's the nearest thing Smooks currently has to an IDE based Debugger (which we hope to have in a future release). It can be very useful for diagnosing issues, or simply as a tool for comprehending a Smooks transformation.
An example HtmlReportGenerator report can be seen&.
Of course you can also write and use your own&&implementations.
Sometimes you want/need to terminate the Smooks filtering process before reaching the end of a message. This can be done by using a &core:terminate& configuration in the Smooks configuration. This configuration only works for the SAX filter - it doesn't really make sense to add it for DOM.
The following is an example configuration that terminates filtering at the end of the customer fragment of the message:
Global configuration settings are, as the name implies, configuration options that can be set once and be applied to all resources in a configuration.
Smooks supports two types of globals, default properties and global parameters:
Global Configuration Parameters: Every &resource-config& in a Smooks configuration can specify &param& elements for configuration parameters. These parameter values are available at runtime through the&, or are reflectively injected through the&@ConfigParam&annotation. Global Configuration Parameters are parameters that are defined centrally (see below) and are accessible to all runtime components via the&&(Vs the&). More on this in the following sections.
Default Properties: Specify default values for &resource-config& attributes. These defaults are automatically applied to&&when their corresponding &resource-config& does not specify the attribute. More on this in the following section.
Global properties differ from the default properties in that they are not specified on the root element and are not automatically applied to resources.
Global parameters are specified in a&&params&&element:
&param name="xyz.param1"&param1-val&/param&&/params&
Global Configuration Parameters are accessible via the&&e.g.:
public void visitAfter(final
element, final ExecutionContext executionContext) throws SmooksException {
param1 = executionContext.getConfigParameter("xyz.param1", "defaultValueABC");&
Default properties are properties that can be set on the root element of a Smooks configuration and have them applied to all resource configurations in smooks-conf.xml file. For example, if you have a resource configuration file in which all the resource configurations have the same selector value, you could specify a&default-selector=order&to save specifying the selector on on every resource configuration:
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:xsl="http://www.milyn.org/xsd/smooks/xsl-1.1.xsd" default-selector="order"&&
&resource-config&
&resource&com.acme.VisitorA&/resource&
&/resource-config&&
&resource-config&
&resource&com.acme.VisitorB&/resource&
&/resource-config&&&smooks-resource-list&
The following default configuration options are available:
default-selector: Selector that will be applied to all resource-config elements in the smooks configuration file, where a selector is not defined.
default-selector-namespace: The default selector namespace, where a namespace is not defined.
default-target-profile: Default target profile that will be applied to all resources in the smooks configuration file, where a target-profile is not defined.
default-condition-ref: Refers to a global condition by the conditions id. This condition is applied to resources that define an empty "condition" element (i.e. &condition/&) that does not reference a globally defined condition.
Filtering specific configurations are made through the&smooks-core&configuration namespace () introduced in Smooks v1.3. For configuring these options on earlier versions of Smooks,&.
An example configuration:
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:core="http://www.milyn.org/xsd/smooks/smooks-core-1.3.xsd"&&
&core:filterSettings type="SAX" defaultSerialization="true" terminateOnException="true"
readerPoolSize="3" closeSource="true" closeResult="true" rewriteEntities="true" /&&
&!-- Other visitor configs etc... --&&&/smooks-resource-list&
type&Determines the type of processing model that will be used. Either SAX or DOM. Please refer to&&for more information about the processing models. Default is SAX ().
defaultSerialization: Determines if default serialization should be switched on (default "true"). Default serialization being turned on simply tells Smooks to locate a&&(or DOMResult) in the Result objects provided to the&&method and to, by default, serialize all events to that Result. This behavior can be turned off using this global configuration parameter and can be overriden on a per fragment basis by targetting a&&implementation at that fragment that takes ownership of the Result writer (in the case of SAX filtering), or simply modifies the DOM (in the case of DOM filtering). As an example of this, see the&.
terminateOnException: Determines whether an exception should terminate processing (default "true").
closeSource: Close Source instance streams passed to the&&method (default "true"). The exception here is System.in, which will never be closed.
closeResult: Close Result streams passed to the&&method (default "true"). The exception here is System.out and System.err, which will never be closed.
rewriteEntities: Rewrite XML entities when reading and writing (default serialization) XML.
readerPoolSize: Reader Pool Size (default 0). Some Reader implementations are very expensive to create (e.g. Xerces). Pooling Reader instances (i.e. reusing) can result in a huge performance improvement, especially when processing lots of "small" messages. The default value for this setting is 0 (i.e. unpooled - a new Reader instance is created for each message). Configure in line with your applications threading model.
Smooks configurations are easily modularized through use of the&&import&&element. This allows you to split Smooks configurations into multiple reusable configuration files and then compose the top level configurations using the&&import&&element e.g.
&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"&&
&import file="bindings/order-binding.xml" /&
&import file="templates/order-template.xml" /&&&/smooks-resource-list&
You can also inject replacement tokens into the imported configuration by using&&param&&sub-elements on the&&import&. This allows you to make tweaks to the imported configuration.
&!-- Top level configuration... --&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"&&
&import file="bindings/order-binding.xml"&
&param name="orderRootElement"&order&/param&
&/import&&&/smooks-resource-list&&&!-- Imported parameterized bindings/order-binding.xml configuration... --&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:jb="http://www.milyn.org/xsd/smooks/javabean-1.4.xsd"&&
&jb:bean beanId="order" class="org.acme.Order" createOnElement="@orderRootElement@"&
&/jb:bean&&&/smooks-resource-list&
Note how the replacement token injection points are specified using&@tokenname@.
Smooks relies on a "Stream Reader" for generating a stream of SAX events from the Source message data stream. A Stream Reader is a class that implements the&&(or the).
By default, Smooks uses the default XMLReader (), but can be easily configured to read non-XML data Sources by configuring a specialized XMLReader:
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"&&
&reader class="com.acme.ZZZZReader" /&&
Other Smooks resources, e.g. &jb:bean& configs for
binding data from the ZZZZ data stream into Java Objects....
--&&&/smooks-resource-list&
The reader can also be configured with a set of handlers, features and parameters. Here is a full example configuration.
&reader class="com.acme.ZZZZReader"&
&handlers&
&handler class="com.X" /&
&handler class="com.Y" /&
&/handlers&
&features&
&setOn feature="http://a" /&
&setOn feature="http://b" /&
&setOff feature="http://c" /&
&setOff feature="http://d" /&
&/features&
&param name="param1"&val1&/param&
&param name="param2"&val2&/param&
&/params&&/reader&
As you can read in the subchapter of this chapter there are a number of non-XML Readers available with Smooks out of the box.
By default Smooks reads XML data.
To set features on the default XML reader, simply omit the class name from the configuration:
&features&
&setOn feature="http://a" /&
&setOn feature="http://b" /&
&setOff feature="http://c" /&
&setOff feature="http://d" /&
&/features&&/reader&
CSV processing through the CSV Reader is configured through the&&configuration namespace.
A simple/basic configuration.
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.5.xsd"&&
Configure the CSV to parse the message into a stream of SAX events.
&csv:reader fields="firstname,lastname,gender,age,country" separator="|" quote="'" skipLines="1" /&&&/smooks-resource-list&
The above configuration will generate an event stream of the form:
&csv-record&
&firstname&Tom&/firstname&
&lastname&Fennelly&/lastname&
&gender&Male&/gender&
&age&21&/age&
&country&Ireland&/country&
&/csv-record&
&csv-record&
&firstname&Tom&/firstname&
&lastname&Fennelly&/lastname&
&gender&Male&/gender&
&age&21&/age&
&country&Ireland&/country&
&/csv-record&&/csv-set&
Fields can be defined in either of 2 ways:
On the 'fields' attribute of the &csv:reader& configuration (as shown above).
As the first record in the message after setting the 'fieldsInMessage' attribute of the &csv:reader& configuration to 'true'.
The field names must follow the same naming rules like XML element names:
Names can contain letters, numbers, and other characters
Names cannot start with a number or punctuation character
Names cannot start with the letters xml (or XML, or Xml, etc)
Names cannot contain spaces
By setting the&rootElementName&and&recordElementName&attributes you can modify the &csv-set& and &csv-record& element names. The same naming rules apply for these names.
All Flat File based reader configurations (including the CSV reader) support&Multi Record Field Definitions, which means that the reader can support CSV message streams that contain varying (multiple different types) CSV record types.
Take the following CSV message example:
book,22 Britannia Road,Amanda Hodgkinsonmagazine,Time,April,2011magazine,Irish Garden,Jan,2011book,The Finkler Question,Howard Jacobson
In this stream, we have 2 record types of "book" and "magazine". We configure the CSV reader to process this stream as follows:
&csv:reader fields="book[name,author] | magazine[*]" rootElementName="sales" indent="true" /&
This reader configuration will generate the following output for the above sample message:
&book number="1"&
&name&22 Britannia Road&/name&
&author&Amanda Hodgkinson&/author&
&magazine number="2"&
&field_0&Time&/field_0&
&field_1&April&/field_1&
&field_2&2011&/field_2&
&/magazine&
&magazine number="3"&
&field_0&Irish Garden&/field_0&
&field_1&Jan&/field_1&
&field_2&2011&/field_2&
&/magazine&
&book number="4"&
&name&The Finkler Question&/name&
&author&Howard Jacobson&/author&
&/book&&/sales&
Note the sytax in the 'fields' attribute. Each record definition is separated by the pipe character '|'. Each record definition is constructed as&record-name[field-name,field-name].&record-name&is matched against the first field in the incoming message and so used to select the appropriate recodr definition to be used for outputting that record. Also note how you can use an astrix character ('*') when you don't want to name the record fields. In this case (as when extra/unexpected fields are present in a record), the reader will generate the output field elements using a generated element name e.g. "field_0", "field_1" etc. See the "magazine" record in the previous example.
Note: Multi Record Field Definitions are not supported when the fields are defined in the messase (fieldsInMessage="true").
String manipulation functions can be defined per field. These functions are executed before that the data is converted into SAX events. The functions are defined after field name, separated with a question mark.
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.5.xsd"&&
&csv:reader fields="lastname?trim.capitalize,country?upper_case" /&&&/smooks-resource-list&
Take a look at the&&chapter for the available functions and how the functions can be chained.
One or more fields of a CSV record can be ignored by specifying the&$ignore$&token in the fields configuration value. You can specify the number of fields to be ignored simply by following the $ignore$ token with a number e.g. "$ignore$3" to ignore the next 3 fields. "$ignore$+" ignores all fields to the end of the CSV record.
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.5.xsd"&&
&csv:reader fields="firstname,$ignore$2,age,$ignore$+" /&&&/smooks-resource-list&
Smooks v1.2 added support for making the binding of CSV records to Java Objects a very trivial task. You no longer need to use the Javabean Cartridge directly (i.e. Smooks main Java binding functionality).
Note: This feature is not supported for Multi Record Field Definitions (see above), or when the fields are defined in the incoming message (fieldsInMessage="true").
A Persons CSV record set such as:
Tom,Fennelly,Male,4,IrelandMike,Fennelly,Male,2,Ireland
Can be bound to a Person of (no getters/setters):
public class Person {
firstname;
private Gender gender;
private int age;}&public enum Gender {
Using a config of the form:
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.5.xsd"&&
&csv:reader fields="firstname,lastname,gender,age,country"&
&!-- Note how the field names match the property names on the Person class. --&
&csv:listBinding beanId="people" class="org.milyn.csv.Person" /&
&/csv:reader&&&/smooks-resource-list&
To execute this configuration:
Smooks smooks = new Smooks(configStream);JavaResult result = new JavaResult();&smooks.filterSource(new StreamSource(csvStream), result);&List&Person& people = (List&Person&) result.getBean("people");
Smooks also supports creation of Maps from the CSV record set:
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.5.xsd"&&
&csv:reader fields="firstname,lastname,gender,age,country"&
&csv:mapBinding beanId="people" class="org.milyn.csv.Person" keyField="firstname" /&
&/csv:reader&&&/smooks-resource-list&
The above configuration would produce a Map of Person instances, keyed by the "firstname" value of each Person. It would be executed as follows:
Smooks smooks = new Smooks(configStream);JavaResult result = new JavaResult();&smooks.filterSource(new StreamSource(csvStream), result);&Map&String, Person& people = (Map&String, Person&) result.getBean("people");&Person tom = people.get("Tom");Person mike = people.get("Mike");
&are also supported, so you can define the&class&attribute as a java.util.Map and have the CSV field values bound into Map instances, which are in turn added to a List or a Map.
Programmatically configuring the CSV Reader on a Smooks instance is trivial (i.e. no XML required). A number of options are available.
The following code configures a Smooks instance with a CSVReader for reading a people record set (see above), binding the record set into a List of Person instances:
Smooks smooks = new Smooks();&smooks.setReaderConfig(new CSVReaderConfigurator("firstname,lastname,gender,age,country")
.setBinding(new CSVBinding("people", Person.class, CSVBindingType.LIST)));&JavaResult result = new JavaResult();smooks.filterSource(new StreamSource(csvReader), result);&List&Person& people = (List&Person&) result.getBean("people");
Of course configuring the Java Binding is totally optional. The Smooks instance could instead (or in conjunction with) be programmatically configured with other&&implementations for carrying out other forms of processing on the CSV record set.
If you're just interested in binding CSV Records directly onto a List or Map of a Java type that reflects the data in your CSV records, then you can use the CSVListBinder or CSVMapBinder classes.
CSVListBinder:
// Note: The binder instance should be cached and reused...CSVListBinder binder = new CSVListBinder("firstname,lastname,gender,age,country", Person.class);&List&Person& people = binder.bind(csvStream);
CSVMapBinder:
// Note: The binder instance should be cached and reused...CSVMapBinder binder = new CSVMapBinder("firstname,lastname,gender,age,country", Person.class, "firstname");&Map&String, Person& people = binder.bind(csvStream);
If you need more control over the binding process, revert back to the lower level APIs:
Fixed Length processing through the Fixed Length Reader is configured through the&&configuration namespace.
A simple/basic configuration.
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:fl="http://www.milyn.org/xsd/smooks/fixed-length-1.3.xsd"&&
Configure the Fixed length to parse the message into a stream of SAX events.
&fl:reader fields="firstname[10],lastname[10],gender[1],age[2],country[2]" skipLines="1" /&&&/smooks-resource-list&
Example input file.
#HEADERTom
M 21 IEMaurice
The above configuration will generate an event stream of the form:
&firstname&Tom
&/firstname&
&lastname&Fennelly
&/lastname&
&gender&M&/gender&
&age& 21&/age&
&country&IE&/country&
&firstname&Maurice
&/firstname&
&lastname&Zeijen
&/lastname&
&gender&M&/gender&
&age&27&/age&
&country&NL&/country&
&/record&&/set&
Fields are defined in the 'fields' attribute as a comma separated list of names and field lengths. The field lengths must be defined between the brackets after the field name (see the example above).
The field names must follow the same naming rules like XML element names:
Names can contain letters, numbers, and other characters
Names cannot start with a number or punctuation character
Names cannot start with the letters xml (or XML, or Xml, etc)
Names cannot contain spaces
By setting the&rootElementName&and&recordElementName&attributes you can modify the &csv-set& and &csv-record& element names. The same naming rules apply for these names.
String manipulation functions can be defined per field. These functions are executed before that the data is converted into SAX events. The functions are defined after the field length definitiona and are optionally separated with a question mark.
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:fl="http://www.milyn.org/xsd/smooks/fixed-length-1.3.xsd"&&
Configure the fixed length reader to parse the message into a stream of SAX events.
&fl:reader fields="firstname[10]?trim,lastname[10]trim.capitalize,gender[1],age[2],country[2]" skipLines="1" /&&&/smooks-resource-list&
Take a look at the&&chapter for the available functions and how the functions can be chained.
Characters ranges of a fixed length record can be ignored by specifying the&$ignore$[10]&token in the fields configuration value. You must specify the number of characters that need be ignored, just as a normal field.
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:fl="http://www.milyn.org/xsd/smooks/fixed-length-1.3.xsd"&&
&fl:reader fields="firstname,$ignore$[2],age,$ignore$[10]" /&&&/smooks-resource-list&
Smooks v1.2 has added support for making the binding of fixed length records to Java Objects a very trivial task. You don't need to use the Javabean Cartridge directly (i.e. Smooks main Java binding functionality).
A Persons fixed length record set such as:
M 21 IEMaurice
Can be bound to a Person of (no getters/setters):
public class Person {
firstname;
private int age;}
Using a config of the form:
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:fl="http://www.milyn.org/xsd/smooks/fixed-length-1.3.xsd"&&
&fl:reader fields="firstname[10]?trim,lastname[10]?trim,gender[1],age[3]?trim,country[2]"&
&!-- Note how the field names match the property names on the Person class. --&
&fl:listBinding beanId="people" class="org.milyn.fixedlength.Person" /&
&/fl:reader&&&/smooks-resource-list&
To execute this configuration:
Smooks smooks = new Smooks(configStream);JavaResult result = new JavaResult();&smooks.filterSource(new StreamSource(fixedLengthStream), result);&List&Person& people = (List&Person&) result.getBean("people");
Smooks also supports creation of Maps from the fixed length record set:
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:fl="http://www.milyn.org/xsd/smooks/fixed-length-1.3.xsd"&&
&fl:reader fields="firstname[10]?trim,lastname[10]?trim,gender[1],age[3]?trim,country[2]"&
&fl:mapBinding beanId="people" class="org.milyn.fixedlength.Person" keyField="firstname" /&
&/fl:reader&&&/smooks-resource-list&
The above configuration would produce a Map of Person instances, keyed by the "firstname" value of each Person. It would be executed as follows:
Smooks smooks = new Smooks(configStream);JavaResult result = new JavaResult();&smooks.filterSource(new StreamSource(fixedLengthStream), result);&Map&String, Person& people = (Map&String, Person&) result.getBean("people");&Person tom = people.get("Tom");Person mike = people.get("Maurice");
&are also supported, so you can define the&class&attribute as a java.util.Map and have the fixed length field values bound into Map instances, which are in turn added to a List or a Map.
Programmatically configuring the FixedLengthReader on a Smooks instance is trivial (i.e. no XML required). A number of options are available.
The following code configures a Smooks instance with a FixedLengthReader for reading a people record set (see above), binding the record set into a List of Person instances:
Smooks smooks = new Smooks();&smooks.setReaderConfig(new FixedLengthReaderConfigurator("firstname[10]?trim,lastname[10]?trim,gender[1],age[3]?trim,country[2]")
.setBinding(new FixedLengthBinding("people", Person.class, FixedLengthBindingType.LIST)));&JavaResult result = new JavaResult();smooks.filterSource(new StreamSource(fixedLengthStream), result);&List&Person& people = (List&Person&) result.getBean("people");
Of course configuring the Java Binding is totally optional. The Smooks instance could instead (or in conjunction with) be programmatically configured with other&&implementations for carrying out other forms of processing on the fixed length record set.
If you're just interested in binding fixed length Records directly onto a List or Map of a Java type that reflects the data in your fixed length records, then you can use the FixedLengthListBinder or FixedLengthMapBinder classes.
FixedLengthListBinder:
// Note: The binder instance should be cached and reused...FixedLengthListBinder binder = new FixedLengthListBinder("firstname[10]?trim,lastname[10]?trim,gender[1],age[3]?trim,country[2]", Person.class);&List&Person& people = binder.bind(fixedLengthStream);
FixedLengthMapBinder:
// Note: The binder instance should be cached and reused...FixedLengthMapBinder binder = new FixedLengthMapBinder("firstname[10]?trim,lastname[10]?trim,gender[1],age[3]?trim,country[2]", Person.class, "firstname");&Map&String, Person& people = binder.bind(fixedLengthStream);
If you need more control over the binding process, revert back to the lower level APIs:
EDI processing in Smooks supported through the&&configuration namespace.
The following is a simple/basic configuration:
&?xml version="1.0"?&&smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:edi="http://www.milyn.org/xsd/smooks/edi-1.2.xsd"&
Configure the EDI Reader to parse the message stream into a stream of SAX events.
&edi:reader mappingModel="edi-to-xml-order-mapping.xml" validate="false"/&&/smooks-resource-list&
mappingModel: Defines the EDI Mapping Model configuration for processing the EDI message stream to a stream of SAX events that can be processed by Smooks.
validate: This attribute turns on/off datatype validation in the EDI Parser. Validation is on by default. It makes sense to turn datatype validation off on the EDI Reader if the EDI data is being bound into a Java Object model (using Java Bindings ala &jb:bean&). This is because the validation will be happening anyway at the binding level.
The EDI to SAX Event mapping is performed based on an "Mapping Model" supplied to the EDI Reader. This model must be based on the&&schema. From this schema, you can see that segment groups are supported (nested segments), including groups within groups, repeating segments and repeating segment groups. Be sure to review the schema.
The following illustration attempts to create a visualisation of the mapping process. The "input-message.edi" file specifies the EDI input, "edi-to-xml-order-mapping.xml" describes how to map that EDI message to SAX events and "expected.xml" illustrates the XML event stream that would result from applying the mapping.
So the above illustration attempts to highlight the following:
How the message delimiters (segment, field, component and sub-component) are specified in the mapping. In particular, how special characters like the linefeed character are specified using XML Character References.
How segment groups (nested segments) are specified. In this case the first 2 segments are part of a group.
How the actual field, component and sub-component values are specified and mapped to the target SAX events (to generate the XML).
What's not shown above is how the &medi:segment& element supports the 2 optional attributes "minOccurs" and "maxOccurs" (default value of 1 in both cases). These attributes can be used to control the optional and required characteristics of a segment. A maxOccurs value of -1 indicates that the segment can repeat any number of times in that location of the EDI message (unbounded).
Segment groups can be added using the &segmentGroup& element. A Segment group is matched by the first segment in the group. A Segment Group can contain nested &segmentGroup& elements, but the first element in a &segmentGroup& must be a &segment&. &segmentGroup& elements support minOccurs/maxOccurs cardinality. They also support an optional "xmlTag" attribute, when if present will result in the XML generated by a matched segment group being inserted inside an element having the name of the xmlTag attribute value.
Segments are matched in one of 2 ways:
By an exact match on the segment code (segcode).
By a&&on the full segment, where the segcode attribute defines the regex pattern (e.g. segcode="1A\*a.*").
required: &field&, &component& and &sub-component& configurations support a "required" attribute, which flags that &field&, &component& or &sub-component& as requiring a value.
By default, values are not required (fields, components and sub-components).
truncatable: &segment&, &field& and &component& configurations support a "truncatable" attribute. For a segment, this means that parser errors will not be generated when that segment does not specify trailing fields that are not "required" (see "required" attribute above). Likewise for fields/components and components/sub-components.
By default, segments, fields, and components are not truncatable.
So, a &field&, &component& and &sub-component& can be present in a message in one of the following states:
Present with a value&(required="true")
Present without a value&(required="false")
Not Present&(required="false" and truncatable="true")
Many message groups use the same segment definitions. Being able to define these segments once and import them into a top level configuration saves on a lot of duplication. A simple configuration demonstrating the import feature would be as follows:
&?xml version="1.0" encoding="UTF-8"?&&medi:edimap xmlns:medi="http://www.milyn.org/schema/edi-message-mapping-1.2.xsd"&&
&medi:import truncatableSegments="true" truncatableFields="true" truncatableComponents="true" resource="example/edi-segment-definition.xml" namespace="def"/&&
&medi:description name="DVD Order" version="1.0"/&&
&medi:delimiters segment="&#10;" field="*" component="^" sub-component="~" escape="?"/&&
&medi:segments xmltag="Order"&
&medi:segment minOccurs="0" maxOccurs="1" segref="def:HDR" segcode="HDR" xmltag="header"/&
&medi:segment minOccurs="0" maxOccurs="1" segref="def:CUS" segcode="CUS" xmltag="customer-details"/&
&medi:segment minOccurs="0" maxOccurs="-1" segref="def:ORD" segcode="ORD" xmltag="order-item"/&
&/medi:segments&&&/medi:edimap&
The configuration-example above demonstrates the use of import that were introduced in Smooks v1.1, where single segments or segments containing childsegments can be separated into a separate file for better reuse in the future.
segref: Contains a "namespace:name" referencing the segment to import.
truncatableSegments: Overrides the 'truncatableSegments' specified in the imported resource mapping file.
truncatableFields: Overrides the 'truncatableFields' specified in the imported resource mapping file.
truncatableComponents: Overrides the 'truncatableComponets' specified in the imported resource mapping file.
Since version 1.2, the &field&, &component& and &sub-component& elements support a "type" attribute that allows datatype specificaton. It actually consists of 2 attributes:
type: The type attribute specifies the basic datatype.
typeParameters: The typeParameters attribute specifies data decoding parameters for the&&associated with the specified type.
The following example shows the type support:
&?xml version="1.0" encoding="UTF-8"?&&medi:edimap xmlns:medi="http://www.milyn.org/schema/edi-message-mapping-1.2.xsd"&&
&medi:description name="Segment Definition DVD Order" version="1.0"/&&
&medi:delimiters segment="&#10;" field="*" component="^" sub-component="~" escape="?"/&&
&medi:segments xmltag="Order"&&
&medi:segment segcode="HDR" xmltag="header"&
&medi:field xmltag="order-id"/&
&medi:field xmltag="status-code" type="Integer"/&
&medi:field xmltag="net-amount" type="BigDecimal"/&
&medi:field xmltag="total-amount" type="BigDecimal"/&
&medi:field xmltag="tax" type="BigDecimal"/&
&medi:field xmltag="date" type="Date" typeParameters="format=yyyyHHmm"/&
&/medi:segment&&
&/medi:segments&&&/medi:edimap&
This type system has a number of uses:
Field Validation.
Programmatically configuring the Smooks instance to use the EDIReader is done through the&:
Smooks smooks = new Smooks();&// Create and initialise the Smooks config for the parser...smooks.setReaderConfig(new EDIReaderConfigurator("/edi/models/invoice.xml"));&// Use the smooks as normalsmooks.filterSource(....);
EJC makes the process of going from EDI to Java much simpler. EJC is similar to&&XJC, accept it is for EDI messages.
EJC generates:
A Java Object model for a given EDI Mapping Model.
A Smooks Java Binding config to populate the Java Object model from an instance of the EDI message described by the EDI Mapping Model (see #1 above).
A Factory class that makes it very east to use EJC to bind EDI data to Java Object Model.
EJC allows you to write simple Java code such as the following:
// Create an instance of the EJC generated Factory class.
This should normally be cached and reused...OrderFactory orderFactory = OrderFactory.getInstance();&// Bind the EDI message stream data into the EJC generated Order model...Order order = orderFactory.fromEDI(ediStream);&// Process the order data...Header header = order.getHeader(); name = header.getCustomerDetails().getName();List&OrderItem& orderItems = order.getOrderItems();
EJC can be executes through Maven or Ant.
Executing the Maven Plugin for EJC is very simple. You just need to install the plugin in your POM file as follows:
&groupId&org.milyn&/groupId&
&artifactId&maven-ejc-plugin&/artifactId&
&version&1.2&/version&
&configuration&
&ediMappingFile&edi-model.xml&/ediMappingFile&
&packageName&com.acme.order.model&/packageName&
&/configuration&
&executions&
&execution&&goals&&goal&generate&/goal&&/goals&&/execution&
&/executions&
&/plugins&&/build&
The plugin has 3 configuration parameters:
ediMappingFile: The path to the&&File, within the maven project. (Optional&- default "src/main/resources/edi-model.xml").
packageName: The Java package into which the generated Java Artifacts are to be located (Java Object Model and Factory class).
destDir: The destination directory in which the generated artifacts are created and compiled from. (Optional&- default "target/ejc").
Executing EJC from an Ant script is trivial. Just configure the EJC Ant task and execute it:
&target name="ejc"&&
&taskdef resource="org/milyn/ejc/ant/anttasks.properties"&
&classpath&&fileset dir="/smooks-1.2/lib" includes="*.jar"/&&/classpath&
&/taskdef&&
&ejc edimappingmodel="src/main/resources/edi-model.xml"
destdir="src/main/java"
pac