The invention claimed is:1. An image processing apparatus comprising:a filter processing section configured to perform a filter process for applying an offset to pixels of a decoded image; anda filter controller configured toinhibit the offset from being applied to an edge portion depending on occurrence of a transform unit in a transformation skip mode in which orthogonal transformation is not performed on a predicted residual, andset a weight depending on a size of the transform unit,wherein the filter controller inhibits the offset from being applied to the edge portion in case an evaluation value calculated using a number of transform units in the transformation skip mode in a maximum coding unit and the weight corresponding to each transform unit of the transform units in the transformation skip mode is equal to or larger than a threshold value, andwherein the filter processing section and the filter controller are each implemented via at least one processor.2. The image processing apparatus according to claim 1, whereinthe filter controller further inhibits the offset from being applied to the edge portion in case the transform unit in the transformation skip mode is included in a maximum coding unit.3. The image processing apparatus according to claim 1, whereinthe filter controller further inhibits the offset from being applied to the edge portion in case a ratio of the number of transform units in the transformation skip mode to a number of transform units in the maximum coding unit is equal to or larger than a threshold value.4. The image processing apparatus according to claim 1, whereinthe filter controller further inhibits the offset from being applied to the edge portion in case a ratio of the number of pixels of transform units in the transformation skip mode to a number of pixels of the maximum coding unit is equal to or larger than a threshold value.5. The image processing apparatus according to claim 1, whereinthe filter controller further inhibits the offset from being applied to the edge portion in case the decoded image includes an image having a high dynamic range.6. The image processing apparatus according to claim 1, whereinthe filter controller is further configured todetermine whether an offset for gradation adjustment using the decoded image is to be applied in the filter processing section or not, andset an offset in case the offset is to be applied.7. The image processing apparatus according to claim 1, further comprising:a prediction selector configured to select a predicted image of either an intra-prediction or an inter-prediction,wherein the prediction selector selects the predicted image of the inter-prediction to calculate the predicted residual in case the transformation skip mode is selected in the inter-prediction, andwherein the prediction selector is implemented via at least one processor.8. The image processing apparatus according to claim 7, whereinthe prediction selector further selects the predicted image of either the intra-prediction or the inter-prediction on a basis of a costs of the inter-prediction and a cost of the intra-prediction in case the transformation skip mode is not selected in the inter-prediction.9. The image processing apparatus according to claim 7, whereinthe prediction selector further selects the predicted image of the inter-prediction in case the transformation skip mode is selected for a transform unit of 4×4 pixels in the inter-prediction.10. The image processing apparatus according to claim 1, whereinthe predicted residual represents a difference between a predicted image generated on a basis of an image as a target to be coded and the decoded image, and the image as the target to be coded.11. The image processing apparatus according to claim 1, whereinthe decoded image includes an image produced by adding a predicted residual generated from a coded stream and a predicted image generated by a prediction process.12. An image processing method comprising:performing a filter process for applying an offset to pixels of a decoded image;inhibiting the offset from being applied to an edge portion depending on occurrence of a transform unit in a transformation skip mode in which orthogonal transformation is not performed on a predicted residual; andsetting a weight depending on a size of the transform unit,wherein the offset is inhibited from being applied to the edge portion in case an evaluation value calculated using a number of transform units in the transformation skip mode in a maximum coding unit and the weight corresponding to each transform unit of the transform units in the transformation skip mode is equal to or larger than a threshold value.